Markovian stochastic approximation with expanding projections

TL;DR

This paper develops a theoretical framework for Markovian stochastic approximation with expanding projections, ensuring stability and convergence even with complex Markov noise and random step sizes, applicable to advanced algorithms like EM with particle M-H sampling.

Contribution

It introduces a new stability and convergence analysis for stochastic approximation with expanding projections under Markovian noise, including non-smooth Markov kernels and random step sizes.

Findings

Proves stability and convergence under general conditions.

Handles non-smooth Markov kernels and random step sizes.

Applies theory to EM algorithms with particle M-H sampling.

Abstract

Stochastic approximation is a framework unifying many random iterative algorithms occurring in a diverse range of applications. The stability of the process is often difficult to verify in practical applications and the process may even be unstable without additional stabilisation techniques. We study a stochastic approximation procedure with expanding projections similar to Andrad\'{o}ttir [Oper. Res. 43 (1995) 1037-1048]. We focus on Markovian noise and show the stability and convergence under general conditions. Our framework also incorporates the possibility to use a random step size sequence, which allows us to consider settings with a non-smooth family of Markov kernels. We apply the theory to stochastic approximation expectation maximisation with particle independent Metropolis-Hastings sampling.