Rate of mutual information between coarse-grained non-Markovian variables

TL;DR

This paper develops a numerical method to estimate the mutual information rate between non-Markovian coarse-grained variables in continuous-time Markov processes, providing both computational tools and analytical bounds, with applications to four-state networks.

Contribution

It introduces a numerical approach to estimate the mutual information rate for non-Markovian variables and derives an analytical upper bound for bipartite Markov processes.

Findings

Numerical method successfully estimates the mutual information rate from time series.

Analytical upper bound derived for bipartite Markov transition rates.

Application demonstrated on four-state network models.

Abstract

The problem of calculating the rate of mutual information between two coarse-grained variables that together specify a continuous time Markov process is addressed. As a main obstacle, the coarse-grained variables are in general non-Markovian, therefore, an expression for their Shannon entropy rates in terms of the stationary probability distribution is not known. A numerical method to estimate the Shannon entropy rate of continuous time hidden-Markov processes from a single time series is developed. With this method the rate of mutual information can be determined numerically. Moreover, an analytical upper bound on the rate of mutual information is calculated for a class of Markov processes for which the transition rates have a bipartite character. Our general results are illustrated with explicit calculations for four-state networks.