Langevin dynamics for the probability of finite state Markov processes

TL;DR

This paper introduces Wasserstein noise perturbations to finite state Markov processes, deriving new stochastic processes with a Fokker-Planck equation and Gibbs stationary distribution, expanding the understanding of gradient flows in probability simplices.

Contribution

It proposes Wasserstein common noises for finite state Markov processes and defines Wasserstein Q-matrices, leading to a new class of stochastic reversible Markov processes.

Findings

Derivation of the functional Fokker-Planck equation for these processes

Identification of Gibbs distribution as the stationary distribution

Examples demonstrating the processes on a two-point state space

Abstract

We study gradient drift-diffusion processes on a probability simplex set with finite state Wasserstein metrics, namely finite state Wasserstein common noises. A fact is that the Kolmogorov transition equation of finite reversible Markov processes satisfies the gradient flow of entropy in finite state Wasserstein space. This paper proposes to perturb finite state Markov processes with Wasserstein common noises. In this way, we introduce a class of stochastic reversible Markov processes. We also define stochastic transition rate matrices, namely Wasserstein Q-matrices, for the proposed stochastic Markov processes. We then derive the functional Fokker-Planck equation in the probability simplex, whose stationary distribution is a Gibbs distribution of entropy functional in a simplex set. Several examples of Wasserstein drift-diffusion processes on a two-point state space are presented.