Stochastic approximation on non-compact measure spaces and application to measure-valued P\'olya processes

TL;DR

This paper proves almost-sure convergence of stochastic approximation algorithms on non-compact measure spaces, applying the results to measure-valued Pólya processes and extending their analysis to weighted and non-balanced cases.

Contribution

It introduces a novel approach using Foster-Lyapunov criteria to generalize stochastic approximation to measure-valued Markov processes on non-compact spaces, enabling new convergence results.

Findings

Almost-sure convergence of a broad class of MVPPs.

Extension of MVPPs to include weights for different colors.

Connection between non-balanced MVPPs and quasi-stationary distributions.

Abstract

Our main result is to prove almost-sure convergence of a stochastic-approximation algorithm defined on the space of measures on a non-compact space. Our motivation is to apply this result to measure-valued P\'olya processes (MVPPs, also known as infinitely-many P\'olya urns). Our main idea is to use Foster-Lyapunov type criteria in a novel way to generalize stochastic-approximation methods to measure-valued Markov processes with a non-compact underlying space, overcoming in a fairly general context one of the major difficulties of existing studies on this subject. From the MVPPs point of view, our result implies almost-sure convergence of a large class of MVPPs, this convergence was only obtained until now for specific examples, with only convergence in probability established for general classes. Furthermore, our approach allows us to extend the definition of MVPPs by adding "weights" to the different colors of the infinitely-many-color urn. We also exhibit a link between non-"balanced" MVPPs and quasi-stationary distributions of Markovian processes, which allows us to treat, for the first time in the literature, the non-balanced case. Finally, we show how our result can be applied to designing stochastic-approximation algorithms for the approximation of quasi-stationary distributions of discrete- and continuous-time Markov processes on non-compact spaces.