Nearest Reversible Markov Chains with Sparsity Constraints: An Optimization Approach

TL;DR

This paper introduces an optimization-based method to approximate non-reversible Markov chains with sparse reversible ones, enabling better modeling in MCMC, chemistry, and data analysis.

Contribution

It formulates the reversible approximation as a quadratic programming problem focusing on sparsity constraints, providing a new principled approach.

Findings

Effective numerical results demonstrate the approach's practicality.

The method enforces reversibility and sparsity simultaneously.

Applications show improved modeling in various domains.

Abstract

Reversibility is a key property of Markov chains, central to algorithms such as Metropolis-Hastings and other MCMC methods. Yet many applications yield non-reversible chains, motivating the problem of approximating them by reversible ones with minimal modification. We formulate this task as a matrix nearness problem and focus on the practically relevant case of sparse transition matrices. The resulting optimization problem is a quadratic programming problem, and numerical experiments illustrate the effectiveness of the approach. This framework provides a principled way to enforce reversibility and sparsity patterns in Markov chains with applications in MCMC, computational chemistry, and data-driven modeling.