The derivation of Markov processes that violate detailed balance

TL;DR

This paper presents a method to derive extended Markov models that include driving agents, allowing cyclic Markov processes violating detailed balance to be understood as approximations of larger equilibrium systems, and clarifies entropy production.

Contribution

It introduces a way to construct extended Markov models with explicit driving degrees of freedom, linking non-equilibrium cyclic processes to equilibrium frameworks.

Findings

Extended Markov models recover cyclic Markov processes as early-time approximations.

Explicit expression of entropy production as a time derivative in the extended model.

Analytic formula for the hidden entropy component in cyclic Markov models.

Abstract

Time-reversal symmetry of microscopic laws dictates that the equilibrium distribution of a stochastic process must obey the detailed balance. On the other hand, cyclic Markov processes that do not admit equilibrium distributions with detailed balance, are often used to model open systems driven out of equilibrium by external agents. I show that for a Markov model without detailed balance, an extended Markov model that explicitly includes the degrees of freedom for the driving agent can be constructed, such that the original cyclic Markov model for the driven system can be recovered as an approximation at early times, by summing over the degrees of freedom for the driving agent. In the process, the widely accepted formula for the entropy production in a cyclic Markov model is explicitly expressed as a time derivative of an entropy component in the extended model. I also find an analytic formula for the entropy component that is hidden in the cyclic Markov model.