Reversible Markov chain estimation using convex-concave programming

TL;DR

This paper introduces a convex-concave reformulation for estimating reversible Markov chains, enabling efficient computation and extensions for various inference scenarios, significantly improving speed over traditional methods.

Contribution

It presents a novel convex-concave reformulation and an efficient primal-dual interior point method for reversible Markov chain estimation, including extensions for additional information and applications.

Findings

Significant speed-up over fixed-point iteration methods.

Effective handling of additional stationary vector information.

Applicability to Markov chain reweighting and inference tasks.

Abstract

We present a convex-concave reformulation of the reversible Markov chain estimation problem and outline an efficient numerical scheme for the solution of the resulting problem based on a primal-dual interior point method for monotone variational inequalities. Extensions to situations in which information about the stationary vector is available can also be solved via the convex- concave reformulation. The method can be generalized and applied to the discrete transition matrix reweighting analysis method to perform inference from independent chains with specified couplings between the stationary probabilities. The proposed approach offers a significant speed-up compared to a fixed-point iteration for a number of relevant applications.