A Riemannian Optimization Approach for Finding the Nearest Reversible Markov Chain

TL;DR

This paper introduces a Riemannian optimization method to efficiently find the closest reversible Markov chain to a given chain, ensuring the same stationary distribution and satisfying detailed balance conditions.

Contribution

It proposes a novel Riemannian optimization framework on a modified multinomial manifold for computing the nearest reversible Markov chain, improving upon existing quadratic programming approaches.

Findings

The method outperforms quadratic programming in synthetic experiments.

It successfully constructs reversible Markov chains from transition count data.

Demonstrates effectiveness in applications involving stochastic differential equations.

Abstract

We address the algorithmic problem of determining the reversible Markov chain $\tilde X$ that is closest to a given Markov chain $X$, with an identical stationary distribution. More specifically, $\tilde X$ is the reversible Markov chain with the closest transition matrix, in the Frobenius norm, to the transition matrix of $X$. To compute the transition matrix of $\tilde X$, we propose a novel approach based on Riemannian optimization. Our method introduces a modified multinomial manifold endowed with a prescribed stationary vector, while also satisfying the detailed balance conditions, all within the framework of the Fisher metric. We evaluate the performance of the proposed approach in comparison with an existing quadratic programming method and demonstrate its effectiveness through a series of synthetic experiments, as well as in the construction of a reversible Markov chain from transition count data obtained via direct estimation from a stochastic differential equation.