Entropy production and fluctuation theorems for active matter

TL;DR

This paper extends stochastic thermodynamics to active matter, defining thermodynamic quantities and fluctuation relations for non-equilibrium active systems, and demonstrates these concepts through numerical simulations.

Contribution

It introduces a framework for thermodynamic analysis of active matter, including definitions of work, heat, and entropy production, and generalizes fluctuation theorems and inequalities.

Findings

Defined thermodynamic quantities for active Ornstein-Uhlenbeck models

Derived fluctuation relations applicable to active matter

Numerical validation of theoretical results

Abstract

Active biological systems reside far from equilibrium, dissipating heat even in their steady state, thus requiring an extension of conventional equilibrium thermodynamics and statistical mechanics. In this Letter, we have extended the emerging framework of stochastic thermodynamics to active matter. In particular, for the active Ornstein-Uhlenbeck model, we have provided consistent definitions of thermodynamic quantities such as work, energy, heat, entropy, and entropy production at the level of single, stochastic trajectories and derived related fluctuation relations. We have developed a generalization of the Clausius inequality, which is valid even in the presence of the non-Hamiltonian dynamics underlying active matter systems. We have illustrated our results with explicit numerical studies.

Figures

Equations

$$⟨v_{ia}⟩=0,⟨v_{ia}(0)v_{jb}(t)⟩=2{\delta }_{ij}{\delta }_{ab}\frac{D}{\tau }{e}^{-|t|/\tau },$$

$${\dot{\mathbf{x}}}_i=-\mu {\nabla }_i\Phi +{\mathbf{v}}_i.$$

$${\dot{\mathbf{x}}}_i$$

$$\dot{{\mathbf{p}}_i}$$

$$W[\Gamma ]={\int }_0^t\mathrm{d}s\left[\frac{\partial \Psi }{\partial s}-\mu \left(\mathbf{P}\cdot \nabla \right)\nabla \Phi \circ \frac{\mathbf{P}}{m}\right],$$

$$Q^{\text{res}}[\Gamma ]=-{\int }_0^t\mathrm{d}s\sum _{i=1}^N\left(-\frac{{\mathbf{p}}_i}{\tau }+\sqrt{\frac{2}{\mu \beta }}{\mathbf{\eta }}_i\right)\circ \frac{{\mathbf{p}}_i(s)}{m}.$$

$$E(t)-E(0)=W[\Gamma ]-Q^{\text{res}}[\Gamma ].$$

$${\dot{\mathbf{x}}}_i$$

$$\dot{{\mathbf{p}}_i}$$

$$\Sigma [\Gamma ]\equiv k_{\text{B}}\ln \frac{P[\Gamma ]}{P^{\text{r}}[{\Gamma }^{\text{r}}]}.$$

$$\frac{\Sigma [\Gamma ]}{k_{\text{B}}}=\Delta s+\beta Q^{\text{res}}[\Gamma ]+\frac{{\mu }^2\beta }{2}\int {\left(\mathrm{d}\mathbf{P}\cdot \nabla \right)}^2\Phi .$$

$$⟨\Sigma ⟩=k_{\text{B}}\Delta H+\frac{⟨Q^{\text{res}}⟩}{T}+k_{\text{B}}\mu {\int }_0^t\mathrm{d}s⟨{\nabla }^2\Phi ⟩\ge 0,$$

$$⟨e^{-\Sigma /k_{\text{B}}}⟩=\sum _{\Gamma }P[\Gamma ]e^{-\Sigma [\Gamma ]/k_{\text{B}}}=\sum _{{\Gamma }^{\text{r}}}{P}^{\text{r}}[{\Gamma }^{\text{r}}]=1,$$

$$P(\Sigma =\sigma )=e^{\sigma /k_{\text{B}}}{P}^{\text{r}}({\Sigma }^{\text{r}}=-\sigma ),$$

$$⟨e^{-{\int }_0^t\mathrm{d}s\dot{\mathbf{\alpha }}\cdot {\nabla }_{\mathbf{\alpha }}\ln {\rho }^s}⟩=1.$$

$$\Delta H+\beta ⟨Q^{\text{res, ex}}⟩\ge 0.$$

$${\dot{\mathbf{x}}}_i=-\mu {\nabla }_i\Phi +{\mathbf{v}}_i,\tau {\dot{\mathbf{v}}}_i=-{\mathbf{v}}_i+\sqrt{2D}{\mathbf{\eta }}_i.$$

$${\dot{\mathbf{p}}}_i$$

$$=$$

$$\frac{dE}{dt}$$

$$=\dot{\mathbf{P}}\circ \frac{\mathbf{P}}{m}+\frac{\partial \Psi }{\partial t}+\dot{\mathbf{X}}\circ \nabla \Psi$$

$$=\frac{\mathbf{P}}{m}\circ \left[-\nabla \Psi -\frac{1}{\mu }\frac{\mathbf{P}}{m}+\sqrt{\frac{2}{\mu \beta }}\mathbf{\eta }-\mu (\mathbf{P}\cdot \nabla )\nabla \Phi \right]+\frac{\partial \Psi }{\partial t}+\frac{\mathbf{P}}{m}\circ \nabla \Psi$$

$$=-\frac{\delta Q^{\text{res}}}{dt}+\frac{\partial \Psi }{\partial t}-\mu (\mathbf{P}\cdot \nabla )\nabla \Phi \circ \frac{\mathbf{P}}{m}$$

$$=-\frac{\delta Q^{\text{res}}}{dt}+\frac{\delta W}{dt}.$$

$${\dot{\mathbf{x}}}_i$$

$$\dot{{\mathbf{p}}_i}$$

$${\dot{\mathbf{x}}}_i$$

$$\dot{{\mathbf{p}}_i}$$

$$P[\Gamma ]\propto \rho ({\mathbf{X}}_0,{\mathbf{P}}_0;0)\underset{\delta t\to 0}{\lim }\prod _{n=0}^{t/\delta t-1}\rho (\delta {\mathbf{P}}_n|{\mathbf{X}}_n,{\mathbf{P}}_n;t_n),$$

$$\rho (\delta {\mathbf{P}}_n|{\mathbf{X}}_n,{\mathbf{P}}_n;t_n)=\prod _{i=1}^N\rho (\delta {\mathbf{p}}_{i,n}|{\mathbf{X}}_n,{\mathbf{P}}_n;t_n)$$

$$\frac{\Sigma [\Gamma ]}{k_{\text{B}}}$$

$$\delta {\mathbf{x}}_i=\frac{{\mathbf{p}}_i}{m}\delta t,$$

Full text

Active matter systems are composed of constitutive elements that are capable of self-propulsion. Either through an internal mechanism or by extracting energy from their environment, these elements exhibit self-induced motion in the absence of any externally applied force. Examples include solutions containing single cellular organisms such as bacteria or protozoa, synthetic colloidal systems, and vibrated monolayers of granular matter Howse et al. (2007); Julicher et al. (2007); Narayan et al. (2007); Cates (2012); Palacci et al. (2013). In fact, active matter models have been used to describe flocking, schooling, and herding behavior in animal movement Vicsek et al. (1995); Toner and Tu (1998); Toner et al. (2005); Vicsek and Zafeiris (2012); Cavagna and Giardina (2014). Active systems are attracting growing interest due to their relevance for understanding live biological systems and their potential applications to the design of synthetic colloidal systems with controllable properties Baskaran and Marchetti (2009); Ramaswamy (2010); Romanczuk et al. (2012); Marchetti et al. (2013); Yeomans et al. (2014); Menzel (2015); Bechinger et al. (2016). Moreover, they exhibit novel collective properties such as phase separation in the absence of explicit attractive interactions Tailleur and Cates (2008); Fily and Marchetti (2012); Buttinoni et al. (2013); Redner et al. (2013); Stenhammar et al. (2013, 2015); Cates and Tailleur (2015a); Redner et al. (2016), rectification of random fluctuations Angelani et al. (2009), and spontaneous self-organization and pattern formation Lambert et al. (2010); van Zuiden et al. (2016), which make their study of interest in its own right.

Active matter systems constitute a new class of condensed matter systems that are inherently out of equilibrium and thereby not describable by the standard, Gibbsian framework. While the collective behavior of active particles has been modeled by hydrodynamic equations based on conservation and symmetry principles Marchetti et al. (2013), and individual active particles by various Brownian dynamics Bechinger et al. (2016), a systematic framework for the nonequilibrium thermodynamics and statistical mechanics of active matter is still in development. Many of the published studies so far have focused on utilizing equilibrium thermostatic notions, often based on approximating active systems by passive systems Lau and Lubensky (2009); Speck et al. (2014); Solon et al. (2015); Bialké et al. (2015); Maggi et al. (2015); Trefz et al. (2016); Speck and Jack (2016); Nikola et al. (2016); Fily et al. (2016). In this Letter we propose an alternative approach based on stochastic thermodynamics Jarzynski (2011); Seifert (2012), which is an emerging framework for the description of thermodynamics and statistical mechanics of stochastic systems far from equilibrium. Stochastic thermodynamics has enabled us to define thermodynamic quantities such as heat, energy, work, entropy, and entropy production at the level of individual realizations of stochastic dynamics. Moreover, one obtains exact analytical results for the fluctuations of entropy production in the form of equalities, as opposed to the inequalities of the second law of thermodynamics. These equalities are more popularly known as fluctuation relations Jarzynski (1997); Crooks (1999); Hatano and Sasa (2001); Speck and Seifert (2005); Seifert (2005); Esposito and den Broeck (2010).

In this Letter, we choose the active Ornstein-Uhlenbeck process (AOUP), alternatively called the Gaussian colored noise model, to illustrate our approach Fodor et al. (2016); Flenner et al. (2016); Paoluzzi et al. (2016); Szamel (2016); Nardini et al. (2016); Marconi et al. (2017). Like other active matter models, this model is known to exhibit motility-induced phase separation (MIPS) Cates and Tailleur (2015b). The AOUP model is different from many other models of active matter Ganguly and Chaudhuri (2013); Chaudhuri (2014, 2016) in that there is no explicit internal drive that forces the system out of equilibrium; the active behavior of the system arises from the nonequilibrium nature of the forces from the environment. In particular, the damping and fluctuating forces from the environment do not satisfy the fluctuation-dissipation relation (FDR). This fact makes it a challenge to develop stochastic thermodynamics for the AOUP as the usual approaches rely heavily on the equilibrium nature of the environment. We overcome this challenge by proposing an exact mathematical mapping of the AOUP, which is an overdamped Langevin model, to a passive, underdamped Langevin model with effective reservoir forces that satisfy the FDR. We derive our generalizations of both the first and the second laws of thermodynamics in reference to this mapped system, the latter giving rise to a modified Clausius inequality. Moreover, we derive both integral and detailed fluctuation relations for entropy production. These in turn allow us to make exact and verifiable predictions for the behavior of the original active matter system.

This Letter is inspired in part by a recent study of the AOUP Fodor et al. (2016), in which the authors studied entropy production of the AOUP based on its path integral representation. We provide an improvement upon their result in that the average entropy production in our treatment (Eq. (9)) is nonzero and positive even for an AOUP in a simple harmonic potential, which is not the case in the former treatment. They report zero average entropy production in such cases. We demonstrate this difference with an explicit example at the end of this Letter. The crucial difference between our treatment and theirs lies in the very definition of entropy production along a trajectory: In our definition (Eq. (8)) we consider the time reversal of the dynamics in addition to the reversal of trajectories, in accordance with the framework of stochastic thermodynamics Seifert (2012); Spinney and Ford (2012); in Fodor et al. (2016) the time reversal of the dynamics had not been considered (see Eq. (9) in Fodor et al. (2016)). Moreover, we have considered a time-dependent scenario in which the potential energy of the system may vary with time — leading to a renormalized potential energy in the underdamped dynamics, different from the original one — a case not considered in the earlier study.

Consider a suspension of $N$ active colloidal particles with $\mathbf{x}_{i}$ denoting the position of the $i$-th particle. In the absence of the medium, the dynamics of the particles are governed by the possibly time-dependent potential $\Phi(\mathbf{X},t)$, where $\mathbf{X}=(\mathbf{x}_{1},\mathbf{x}_{2},\ldots,\mathbf{x}_{N})$ refers to the configuration space of the whole system. There are two forces from the medium: a damping force, $-\dot{\mathbf{x}_{i}}/\mu$ for particle $i$, and a Gaussian random force, $\mathbf{v}_{i}/\mu$, the latter having the following properties Maggi et al. (2015):

[TABLE]

for all $i,j,a$ and $b$. Here, the angular bracket $\langle\ldots\rangle$ denotes the noise average (i.e., the average with respect to many realizations of the random forces $\mathbf{v}_{i}$); $v_{ia}$ denotes the $a$-th component of $\mathbf{v}_{i}$; $\delta_{xy}$ denotes the Kronecker delta function; $\tau$ is the persistence time of the noise; and $D$ is the diffusion coefficient. Equation (1) implies, in particular, that the random forces felt by different particles in different directions are independent of each other. The Langevin equation of the $i$-th particle is given by

[TABLE]

Note that it is possible to write down a Langevin equation for the noises $\mathbf{v}_{i}$ themselves: $\tau\dot{\mathbf{v}}_{i}=-\mathbf{v}_{i}+\sqrt{2D}\boldsymbol{\eta}_{i}$, where $\boldsymbol{\eta}_{i}$ denotes a Gaussian random noise with the properties $\langle\eta_{ia}\rangle=0$ and $\langle\eta_{ia}(0)\eta_{jb}(t\rangle=2\delta_{ij}\delta_{ab}\delta(t)$ where $\delta(t)$ denotes the Dirac delta distribution.

Because the noise force $\mathbf{v}_{i}/\mu$ has exponential memory whereas the damping force $-\dot{\mathbf{x}}_{i}/\mu$ is memoryless, the model violates the FDR, and we have to conclude that the medium is out of equilibrium. As pointed out in Fodor et al. (2016), in the limit of vanishing persistence time ($\tau\rightarrow 0$) the model reduces to an equilibrium model satisfying the FDR with respect to temperature $T\equiv D/(k_{\text{B}}\mu)$ where $k_{\text{B}}$ is the Boltzmann constant. Motivated by this observation, we replace $D$ in the following by $\mu k_{\text{B}}T$. We also use the notation $\beta=1/k_{\text{B}}T$.

As remarked in the introduction, the absence of the FDR implies that we cannot take the medium to be in equilibrium and therefore cannot utilize the framework of stochastic thermodynamics as is. In particular, we cannot interpret the heat given to the medium by $T$ to be the change in entropy of the medium for finite persistence time $\tau$. Fortunately, it is possible to overcome this challenge because of a surprising property of the fluctuations of this active matter system: the overdamped AOUP can be mapped exactly to an underdamped Langevin process where the new, effective medium (reservoir) is in equilibrium. The effective underdamped process is given by (Appendix A)

[TABLE]

where $\mathbf{p}_{i}$ is the auxiliary momentum of particle $i$; $m=\tau/\mu$ is the effective mass; $\Psi=\Phi+\mu m(\partial/\partial t)\Phi$ is an effective potential; $\mathbf{P}=({\mathbf{p}}_{1},\mathbf{p}_{2},\ldots,\mathbf{p}_{N})$ and $\mathbf{X}=(\mathbf{x}_{1},\mathbf{x}_{2},\ldots,\mathbf{x}_{N})$ refer to the phase space of the whole system. Also, we have used $\boldsymbol{\nabla}_{i}$ to denote the gradient with respect to $\mathbf{x}_{i}$ and $\boldsymbol{\nabla}=(\boldsymbol{\nabla}_{1},\boldsymbol{\nabla}_{2},\ldots,\boldsymbol{\nabla}_{N})$ to denote the spatial gradient in the phase space of the whole system. The damping and the noise terms, $-\mathbf{p}_{i}/\mu m$ and $\sqrt{2/\mu\beta}\boldsymbol{\eta}_{i}$, respectively, satisfy the FDR with respect to temperature $T$. In the following we interpret them to be forces from the effective, equilibrium reservoir. The nonequilibrium nature of this mapped dynamics arise from the momentum-dependent force $\mathbf{F}_{i,m}\equiv-\mu\left(\mathbf{P}\cdot\boldsymbol{\nabla}\right)\boldsymbol{\nabla}_{i}\Phi$. In some sense, we have decomposed the forces of the nonequilibrium medium into those of an underlying equilibrium reservoir and explicit forces. We now develop the results of stochastic thermodynamics around this model.

To begin we consider the first law of thermodynamics — namely, conservation of energy. The energy of the system is given by $E=(1/2)P^{2}/m+\Psi$, kinetic plus potential energy. A trajectory $\Gamma$ over the interval $[0,t]$ is defined to be the sequence of points $\Gamma=\{\mathbf{X}_{0:t},\mathbf{P}_{0:t}\}=\{\mathbf{X}(s),\mathbf{P}(s)|0\leq s\leq t\}$. Work done on the system along any $\Gamma$ is given by Sekimoto (2010):

[TABLE]

where the first term denotes the thermodynamic work corresponding to the conservative force, $-\boldsymbol{\nabla}_{i}\Psi$, and the second term denotes the mechanical work done by the nonconservative force, $\mathbf{F}_{i,m}$.111In the terminology of Jarzynski (2007), the first term is the so called inclusive work done on the system corresponding to the time dependence of the potential $\Psi$, and the second term the exclusive work done by the momentum-dependent force $\mathbf{F}_{i,m}$. Here, the circle ($\circ$) denotes Stratonovich multiplication Gardiner (2009), the necessity of which follows from the chain rule of derivatives Sekimoto (1997, 2010). The heat given to the reservoir over $\Gamma$ is the amount of work done against the reservoir forces:

[TABLE]

The intuition behind the formula is the following: the reservoir degrees of freedom are unstructured and random, and any energy given to a random, unstructured medium should be interpreted as heat (as opposed to work) Callen (1985). Consistency of these definitions can be seen through the following relation, the first law of thermodynamics for the current system:

[TABLE]

The derivation is presented in Appendix B.

We now consider the second law of thermodynamics. Let $\rho({\bf X},{\bf P};t)$ be the phase space probability distribution of the system at any time $t$. If the system is at $(\mathbf{X}(t),\mathbf{P}(t))$ at time $t$, following the developments in stochastic thermodynamics we can define the stochastic entropy Seifert (2005) of the system at $t$ to be $s(t)=-\ln{\rho}(\mathbf{X}(t),\mathbf{P}(t);t)$. The change in entropy of the reservoir over the interval $[0,t]$ is the heat given to it divided by $T$ as given by the Clausius formula $\beta Q^{\text{res}}[\Gamma]$ (Eq. (5)). Unlike the usual second law of thermodynamics for passive systems, however, the total entropy production over $[0,t]$ is not just the sum of the change in the stochastic entropy of the system, $\Delta s$, and the Clausius entropy change of the medium, $Q^{\text{res}}/T$. To see this we need to first define the time reversal of the mapped process (Appendix C):

[TABLE]

obtained by keeping the reservoir terms unchanged and replacing $t$ and $\mathbf{P}$ by $-t$ and $-\mathbf{P}$, respectively, in the rest of the terms. We also need to consider the time reversal of the phase space trajectory $\Gamma$, given by $\Gamma^{\text{r}}=\{\mathbf{X}^{\text{r}}_{0:t},\mathbf{P}^{\text{r}}_{0:t}\}$ with $\mathbf{X}^{\text{r}}(s)=\mathbf{X}(t-s)$ and $\mathbf{P}^{\text{r}}(s)=-\mathbf{P}(t-s)$. Next, we need to consider $P(\Gamma)$, the probability of $\Gamma$ in the mapped dynamics and $P^{\text{r}}(\Gamma^{\text{r}})$, the probability of the time-reversed trajectory $\Gamma^{\text{r}}$ in the time-reversed dynamics (Eq. (7)). If there is any time dependence in $\Psi$ and $\Phi$, the time dependence has to be reversed as well in the reverse process. Because entropy production is a measure of time-reversal symmetry breaking, entropy production $\Sigma[\Gamma]$ along $\Gamma$ is given by

[TABLE]

Using Eqs. (3) and (7), we can derive an explicit path integral expression (Appendix D):

[TABLE]

It is easy to prove that the average of $\Sigma$ is non-negative as required by the second law of thermodynamics. This follows from the fact that the average of $\Sigma$ can be written as $\langle\Sigma\rangle/k_{\text{B}}=\sum_{\Gamma}P[\Gamma]\ln{[P[\Gamma]/P^{\text{r}}(\Gamma^{\text{r}})]}$, which takes the form of a relative entropy with the known property that it is never negative amd Joy A. Thomas (2006). In fact, we can derive the following expression for the average entropy production (Appendix E):

[TABLE]

where $H[\rho]=\langle s\rangle=-\int_{\mathbf{X,P}}\rho\ln{\rho}$ is the Shannon entropy of the system. This is the central result of our paper. It expresses the second law of thermodynamics as a modified Clausius inequality. Each quantity in the inequality can be calculated even for the original dynamics as the mapped system is mathematically equivalent to the AOUP. The inequality therefore constitutes a prediction for the original system.

The difference between the usual Clausius inequality and our generalization of the second law described above is embodied by the last term. By comparison with Eq. (9), it is readily apparent that the last term arises from the momentum-dependent force $\mathbf{F}_{i,m}$, though this last term is not the average work done by $\mathbf{F}_{i,m}$ as can be seen from Eq. (4). There is, indeed, an interesting origin of this term stemming from the mapped dynamics, Eq. (3). If the reservoir terms are taken out, due to $\mathbf{F}_{i,m}$, the dynamics is not Hamiltonian and the phase space volume is not conserved. The last term in Eq. (10) is the integral of the average phase space contraction rate due to $\mathbf{F}_{i,m}$ (Appendix F). For deterministic thermostats, this can be interpreted as the entropy production Andrey (1985); Cohen and Rondoni (1998); Dorfman (1999); Ruelle (2003). This is the same quantity that appeared in a recent generalization of the Jarzynski equality for non-Hamiltonian dynamics Mandal and DeWeese (2016). Moreover, recent developments in stochastic thermodynamics have demonstrated that the usual Clausius inequality has to be modified in the presence of feedback control Kim and Qian (2004); Cao and Feito (2009); Sagawa and Ueda (2010); Toyabe et al. (2010); Ponmurugan (2010); Horowitz and Vaikuntanathan (2010); Abreu and Seifert (2012); Lahiri et al. (2012); Kundu (2012); Munakata and Rosinberg (2012); Liu et al. (2014); Parrondo et al. (2015). In particular, there are extra terms in the second law because the external feedback controller is not accounted for by explicit degrees of freedom. The momentum-dependent force $\mathbf{F}_{i,m}$ can be seen as a spatially inhomogeneous feedback cooling operation. The last term in Eq. (10) refers to this extra contribution. Towards the end of this Letter we will demonstrate that it is crucial to include this extra term. In its absence the inequality may not be satisfied.

A major contribution from stochastic thermodynamics is the surprising result that the inequalities of the second law can be replaced by exact equalities. These are in a sense more refined versions of the inequalities because the latter can be derived by the application of Jensen’s inequality. We can derive the following exact relation, the integral fluctuation relation, for entropy production in our active matter system:

[TABLE]

where the second relation follows from the definition of entropy production (Eq. (8)) and the third relation follows from the normalization of $P^{\text{r}}[\Gamma^{r}]$. Equation (11) is our second main result. The modified Clausius inequality $\langle\Sigma\rangle\geq 0$ follows from the application of Jensen’s inequality to Eq. (11). In fact, there is a more detailed equality underlying Eq. (11) (Appendix G):

[TABLE]

where $P(\Sigma=\sigma)$ denotes the probability density of $\Sigma=\sigma$ in the process described by Eq. (3) and $P^{\text{r}}(\Sigma^{\text{r}}=\sigma)$ denotes the same quantity for the reverse process (Eq. (7)). We have to assume that the initial condition for the reverse process is obtained from the final condition of the first process by reversing the sign of the momenta. Equation (12) is the detailed fluctuation relation for entropy production.

There are three qualitatively different fluctuation relations (both integral and detailed) in stochastic thermodynamics for passive, overdamped dynamics: that of total entropy production, excess entropy production, and housekeeping heat. We have extended the relation for total entropy production to the AOUP. For the sake of completeness we will address the latter two cases in the following. The fluctuation relation for excess entropy production is a consequence of the Markovian nature of the dynamics. Let the steady state distribution of the dynamics in Eq. (3) in the absence of any time dependence of $\Phi$ be $\rho^{\text{s}}(\mathbf{X},\mathbf{P};\boldsymbol{\alpha})$, where $\boldsymbol{\alpha}$ represents fixed, external parameters of the system. We then have the following integral fluctuation relation for transitions between any two steady states (characterized by two different values of $\boldsymbol{\alpha}$)

[TABLE]

A proof of this relation follows the same basic steps as that for a related expression for passive Langevin systems, which is presented in Hatano and Sasa (2001). We can rewrite the integral in the exponent as $-\Delta\ln\rho^{\text{s}}+\int_{0}^{t}\mathrm{d}s\,\dot{\mathbf{Z}}\circ\boldsymbol{\nabla_{Z}}\ln{\rho^{\text{s}}}$, with $\mathbf{Z}(s)=(\mathbf{X}(s),\mathbf{P}(s))$. The last quantity, the generalized work done against the generalized (nonequilibrium) force $-k_{\text{B}}T\boldsymbol{\nabla_{Z}}\ln{\rho^{\text{s}}}$, is called the excess heat $Q^{\text{res, ex}}[\Gamma]$. Equation (13) therefore gives the integral fluctuation relation for excess entropy production $\Sigma^{\text{ex}}\equiv\Delta s+Q^{\text{res, ex}}/T$ (in units of $k_{\text{B}}$). Unlike total entropy production, there is generally no detailed fluctuation relation for excess entropy production because the steady state distribution $\rho^{\text{s}}$ is generally not time-reversal symmetric, $\rho^{\text{s}}(\mathbf{X},-\mathbf{P})\neq\rho^{\text{s}}(\mathbf{X},\mathbf{P})$ Lahiri and Jayannavar (2014). Nevertheless, by applying Jensen’s inequality to the integral fluctuation relation (13) we get the following inequality satisfied by the dynamics:

[TABLE]

This can be a stricter bound for $\Delta H$ compared to that of Eq. (10). The concept of housekeeping heat for the current system is ambiguous due to the momentum-dependent force $\mathbf{F}_{i,m}$. It has been recently shown that such systems have many different notions of housekeeping heat each with its own fluctuation relation and consequent inequality Yeo et al. (2016). Given this ambiguity, we reserve a detailed discussion of the relevant results for a future study.

We now illustrate our results with a simple case study. Consider a single active particle in a one-dimensional simple harmonic potential, $\Phi=x^{2}/2$, where we have set the spring constant to one. For simplicity, we assume the other parameters, $\mu$, $\tau$, T, and the constant $k_{\text{B}}$, to be unity as well. In Fig. 1 we have plotted the probability distribution of entropy production $\Sigma$ over 0.5 units of time in the steady state of the system. We see that it is possible to have negative entropy production over individual trajectories, but the average over sufficiently many trajectories is always non-negative. In particular, the average entropy production in the current case turns out to be $\langle\Sigma\rangle=0.250\pm 0.005$. This is in contrast to the entropy production formula proposed in Fodor et al. (2016) where the average entropy production of active Ornstein-Uhlenbeck particles for a quadratic potential $\Phi$ is zero. To show the importance of the phase space contraction term in entropy production, the last term in Eq. (10), we have also measured the incomplete entropy production $\Sigma^{\prime}=\Delta s+Q^{\text{res}}/T$. In this case we find $\langle\Sigma^{\prime}\rangle=-0.250\pm 0.005$. We see that the average is negative implying that the usual Clausius inequality is not satisfied even for an AOUP in one dimension with a simple harmonic potential.

Here we have shown how stochastic thermodynamics can be used to consistently define thermodynamic quantities for active matter Speck (2016). We have demonstrated our method for a particular model of active matter, but we emphasize that our approach is generalizable to other active matter systems. This is an important step in the construction of the thermodynamics and statistical mechanics of active matter, which will be necessary for understanding response functions and material properties of these systems.

Supplementary materials

Appendix A Derivation of Eq. (3)

We start with the equations

[TABLE]

We define momentum by $\mathbf{p}_{i}=m\dot{\mathbf{x}}_{i}$ with $m=\tau/\mu$. The rate of change of $\mathbf{p}_{i}$ is given by

[TABLE]

with $\Psi=\Phi+m\,\mu\,\partial\Phi/\partial t$. In the second line we have used Eq. (15). In the fourth line we have used the chain rule of derivatives and Eq. (15). Because the noise $\mathbf{v}_{i}$ is colored, and not white, there is no Ito correction in the chain rule. In the fifth line we have used Eq. (15) again to replace $\mathbf{v}_{i}$ in terms of other quantities. In the last line we have introduced the renormalized potential $\Psi$ to simplify the formula. The last line corresponds to Eq. (3) in the main text.

Appendix B Derivation of Eq. (6)

We have

[TABLE]

In the second line we have used the chain rule. Because the products are interpreted in the Stratonovich sense we do not have any Ito correction. In the third line we have used the equation of motion (3) and defined $\boldsymbol{\eta}=(\boldsymbol{\eta}_{1},\boldsymbol{\eta}_{2},\ldots,\boldsymbol{\eta}_{N})$. In the fourth line we have used cancellation of terms and the definition of heat given in Eq. (5). The last line is the differential version of Eq. (6).

Appendix C Derivation of Eq. (7)

To derive Eq. (7), we start with Eq. (3) without the reservoir terms:

[TABLE]

The rationale is that the reservoir is already in equilibrium, as it satisfies the FDR, and therefore already time-reversal symmetric in its dynamics. We then use the standard prescription for time reversal of replacing $t$ and $\mathbf{P}$ by $-t$ and $-\mathbf{P}$, respectively. We then find

[TABLE]

Finally we put the reservoir terms back in the equation to get the time-reversed dynamics in Eq. (7). The same approach was used by Pradhan and Seifert Pradhan and Seifert (2010) for studying the time reversal of charged Brownian dynamics in an equilibrium reservoir.

Appendix D Derivation of Eq. (9)

We start with the definition $\Sigma[\Gamma]\equiv k_{\text{B}}\ln{(P[\Gamma]/P^{\text{r}}[\Gamma^{\text{r}}])}$. The path probability $P[\Gamma]$ can be written as the limit

[TABLE]

where $(\mathbf{X}_{0},\mathbf{P}_{0})$ is the initial point of $\Gamma$; $\rho(\mathbf{X}_{0},\mathbf{P}_{0};0)$ is the initial probability density; and $\rho(\delta\mathbf{P}_{n}|\mathbf{X}_{n},\mathbf{P}_{n};t_{n})$ is the probability density of momentum increment $\delta\mathbf{P}_{n}$ over the time interval $(t_{n},t_{n}+\delta t)$, for $t_{n}=n\delta t$, given the position $\mathbf{X}_{n}$ and momentum $\mathbf{P}_{n}$ at time $t_{n}$. The increment in position, $\delta\mathbf{X}_{n}$, is completely determined once the increment in momentum, $\delta\mathbf{P}_{n}$, is specified; therefore, we do not need to consider a probability density over $\delta{\mathbf{X}}_{i}$. Alternatively, we could have used a delta function distribution Ganguly and Chaudhuri (2013). Each probability density $\rho(\delta\mathbf{P}_{n}|\mathbf{X}_{n},\mathbf{P}_{n};t_{n})$ can be written as the product

[TABLE]

because the noise terms of different particles are independent of each other. (See the discussion following Eq. (1)). In terms of $\rho(\delta\mathbf{p}_{i,n}|\mathbf{x}_{i,n},\mathbf{p}_{i,n};t_{n})$, the entropy production can be written as Spinney and Ford (2012)

[TABLE]

with $(\mathbf{X}^{\text{r}}_{0},\mathbf{P}^{\text{r}}_{0})=(\mathbf{X}_{t},-\mathbf{P}_{t})$. If we choose the initial condition $\rho^{\text{r}}(\mathbf{X}^{\text{r}}_{0},\mathbf{P}^{\text{r}}_{0};0)=\rho(\mathbf{X}^{\text{r}}_{0},-\mathbf{P}^{\text{r}}_{0};t)$ the first ratio on the right gives the change in stochastic entropy $\Delta s$ over $\Gamma$. The last term denotes the entropy production in excess of $\Delta s$. In the following, we show that this last term is equal to $\beta Q^{\text{res}}[\Gamma]+\frac{\mu^{2}}{2}\int_{\Gamma}\left[\left(\delta\boldsymbol{P}\cdot\boldsymbol{\nabla}\right)\boldsymbol{\nabla}_{i}\right]\Phi$ in five steps:

express the Langevin equations (3) and (7) in their differential forms; 2. 2.

use the differential forms to derive explict expressions for the probability densities $\rho(\delta\mathbf{p}_{i,n}|\mathbf{x}_{i,n},\mathbf{p}_{i,n};t_{n})$; 3. 3.

evaluate the log ratio $\ln{\left[\frac{\rho(\delta\mathbf{p}_{i,n}|\mathbf{x}_{i,n},\mathbf{p}_{i,n};t_{n})}{\rho^{\text{r}}(\delta\mathbf{p}_{i,n}|\mathbf{x}_{i,n+1},-\mathbf{p}_{i,n+1};t_{n+1})}\right]}$ up to first order in $\delta t$; 4. 4.

evaluate $\delta Q^{\text{res}}$ up to first order in $\delta t$; 5. 5.

compare the above results to arrive at $\prod_{i=1}^{N}\ln{\left[\frac{\rho(\delta\mathbf{p}_{i,t}|\mathbf{X}_{t},\mathbf{P}_{t};t)}{\rho^{\text{r}}(\delta\mathbf{p}_{i,t}|\mathbf{X}_{t+\delta t},-\mathbf{P}_{t+\delta t};t+\delta t)}\right]}=\beta\delta Q^{\text{res}}+\frac{\mu^{2}}{2}\left(\delta\boldsymbol{P}\cdot\boldsymbol{\nabla}\right)^{2}\Phi$ (up to first order in $\delta t$).

The differential forms of the Langevin equations (3) and (7) are given by, respectively,

[TABLE]

with $\boldsymbol{\alpha}_{i}=\boldsymbol{\alpha}_{i}(\mathbf{X},\mathbf{P},t)=\boldsymbol{\nabla}_{i}\Psi+\frac{\mathbf{p}_{i}}{\tau}+\mu(\mathbf{P}\cdot\boldsymbol{\nabla})\boldsymbol{\nabla}_{i}\Phi$ and $\boldsymbol{\alpha}^{\text{r}}_{i}(\mathbf{X},\mathbf{P},t)=\boldsymbol{\nabla}_{i}\Psi+\frac{\mathbf{p}_{i}}{\tau}-\mu(\mathbf{P}\cdot\boldsymbol{\nabla})\boldsymbol{\nabla}_{i}\Phi$. Here, $\mathbf{N}_{i,(0,1)}(t)$ and $\mathbf{N}^{\text{r}}_{i,(0,1)}(t)$ are vectors of random, normal variables. In writing the equations (23) and (24) we have used the Ito interpretation. Because the noise terms are additive, the same final results would also be obtained by using the Stratonovich interpretation.

Using Eqs. (23) and (24) we get the following expressions at any time $t$

[TABLE]

where $G[a,\nu^{2}]$ denotes a Gaussian distribution with mean $a$ and variance $\nu^{2}$.

Now we evaluate the log ratio $\ln{\left[\frac{\rho(\delta\mathbf{p}_{i}|\mathbf{X},\mathbf{P};t))}{\rho^{\text{r}}(\delta\mathbf{p}_{i}|\mathbf{X}+\delta\mathbf{X},-\mathbf{P}-\delta\mathbf{P};t+\delta t)}\right]}$ keeping terms up to order $\delta t$. In the following, we say that two expressions $f$ and $g$ are equivalent, $f\sim g$, if they differ by terms on the order of $(\delta t)^{1+\epsilon}$ with $\epsilon>0$. We get, omitting some arguments for brevity,

[TABLE]

We can simplify the first term in the bracket on the right as

[TABLE]

Similarly, we can simplify the second term as

[TABLE]

Collecting results, and summing over all particles, we find

[TABLE]

An expression for $\delta Q^{\text{me}}$ can be obtained through

[TABLE]

Combining the last two equations, we get the desired result: $\prod_{i=1}^{N}\ln{\left[\frac{\rho(\delta\mathbf{p}_{i,t}|\mathbf{X}_{t},\mathbf{P}_{t};t)}{\rho^{\text{r}}(\delta\mathbf{p}_{i,t}|\mathbf{X}_{t+\delta t},-\mathbf{P}_{t+\delta t};t+\delta t)}\right]}=\beta\delta Q^{\text{res}}+\frac{\mu^{2}\beta}{2}\left(\delta\boldsymbol{P}\cdot\boldsymbol{\nabla}\right)^{2}\Phi$.

Appendix E Derivation of Eq. (10)

We need to show $\left\langle\frac{\mu^{2}\beta}{2}\left(\delta\boldsymbol{P}\cdot\boldsymbol{\nabla}\right)^{2}\Phi|\mathbf{X,P};t\right\rangle\sim\mu\,\delta t\,\nabla^{2}\Phi$. This is best done with indicial notation:

[TABLE]

In the first line we have written the left hand side in indicial notation and used the fact that derivatives of $\Phi$ are known quantities if $\mathbf{X}$ is known; in the second line we have used Eq. (23) and the fact that the noise is independent in different particles and directions; and in the last line we have used the properties of Kronecker delta function to simplify the summation.

Appendix F Phase space contraction

The phase space contraction rate is given by

[TABLE]

where in the third and fourth lines we have used indicial notation.

Appendix G DFT for entropy production

Following are the steps in the proof of the detailed fluctuation relation for entropy production:

[TABLE]

The first line is just the definition of the left hand side in path integral notation; the second line follows from Eq. (8); the third line follows from the initial condition $\rho^{\text{r}}(\mathbf{X}^{\text{r}}_{0},\mathbf{P}^{\text{r}}_{0};0)=\rho(\mathbf{X}^{\text{r}}_{0},-\mathbf{P}^{\text{r}}_{0};t)$; and the last line follows from the definition of the probability density of entropy production, Eq. (34a), applied to the time-reversed process.