Stochastic approximation of quasi-stationary distributions on compact spaces and applications

TL;DR

This paper introduces a recursive algorithm for approximating quasi-stationary distributions of Markov chains on compact spaces, with applications to diffusions and spectral gap estimation, ensuring convergence under general conditions.

Contribution

It extends previous finite Markov chain methods to general Markov chains on compact spaces, providing convergence analysis and practical applications.

Findings

Algorithm converges to the quasi-stationary distribution under broad conditions.

Application to diffusion processes killed at boundary.

Estimation of spectral gaps for irreducible Markov processes.

Abstract

In the continuity of a recent paper ([6]), dealing with finite Markov chains, this paper proposes and analyzes a recursive algorithm for the approximation of the quasi-stationary distribution of a general Markov chain living on a compact metric space killed in finite time. The idea is to run the process until extinction and then to bring it back to life at a position randomly chosen according to the (possibly weighted) empirical occupation measure of its past positions. General conditions are given ensuring the convergence of this measure to the quasi-stationary distribution of the chain. We then apply this method to the numerical approximation of the quasi-stationary distribution of a diffusion process killed on the boundary of a compact set and to the estimation of the spectral gap of irreducible Markov processes. Finally, the sharpness of the assumptions is illustrated through the study of the algorithm in a non-irreducible setting.