Stationary averaging for multi-scale continuous time Markov chains using parallel replica dynamics

TL;DR

This paper introduces two parallel algorithms for efficiently estimating stationary averages of metastable continuous-time Markov chains, leveraging parallel replica dynamics without requiring reversibility, and demonstrates their effectiveness through analysis and simulations.

Contribution

The paper develops and analyzes two novel parallel algorithms for simulating metastable Markov chains, applicable without reversibility assumptions.

Findings

Algorithms accurately estimate stationary averages.

Methods improve efficiency in exploring metastable states.

Numerical simulations confirm consistency and efficiency.

Abstract

We propose two algorithms for simulating continuous time Markov chains in the presence of metastability. We show that the algorithms correctly estimate, under the ergodicity assumption, stationary averages of the process. Both algorithms, based on the idea of the parallel replica method, use parallel computing in order to explore metastable sets more efficiently. The algorithms require no assumptions on the Markov chains beyond ergodicity and the presence of identifiable metastability. In particular, there is no assumption on reversibility. For simpler illustration of the algorithms, we assume that a synchronous architecture is used throughout of the paper. We present error analyses, as well as numerical simulations on multi-scale stochastic reaction network models in order to demonstrate consistency of the method and its efficiency.