Information Landscape and Flux, Mutual Information Rate Decomposition and Entropy Production

TL;DR

This paper investigates the dynamics of information systems, decomposing mutual information rate into reversible and irreversible parts, and linking entropy production to information irreversibility using a bivariate Markov chain example.

Contribution

It introduces a novel decomposition of mutual information rate into reversible and irreversible components, connecting information dynamics with nonequilibrium thermodynamics.

Findings

Mutual information rate can be decomposed into reversible and irreversible parts.

The irreversible part of mutual information relates to entropy production.

Demonstrated the concepts using a bivariate Markov chain model.

Abstract

We explore the dynamics of information systems. We show that the driving force for information dynamics is determined by both the information landscape and information flux which determines the equilibrium time reversible and the nonequilibrium time-irreversible behaviours of the system respectively. We further demonstrate that the mutual information rate between the two subsystems can be decomposed into the time-reversible and time-irreversible parts respectively, analogous to the information landscape-flux decomposition for dynamics. Finally, we uncover the intimate relation between the nonequilibrium thermodynamics in terms of the entropy production rates and the time-irreversible part of the mutual information rate. We demonstrate the above features by the dynamics of a bivariate Markov chain.