Convergence of Markovian Stochastic Approximation with discontinuous dynamics

TL;DR

This paper analyzes the convergence of Markovian stochastic approximation algorithms with discontinuous dynamics, relaxing traditional smoothness assumptions and applying results to quantile estimation and vector quantization.

Contribution

It introduces convergence results for stochastic approximation algorithms with discontinuous functions, extending prior work that required smoothness in the dynamics.

Findings

Convergence established under weak smoothness assumptions.

Application to adaptive quantile estimation.

Application to penalized vector quantization.

Abstract

This paper is devoted to the convergence analysis of stochastic approximation algorithms of the form $\theta\_{n+1} = \theta\_n + \gamma\_{n+1} H\_{\theta\_n}(X\_{n+1})$ where $\{\theta\_nn, n \geq 0\}$ is a $R^d$-valued sequence, $\{\gamma, n \geq 0\}$ is a deterministic step-size sequence and $\{X\_n, n \geq 0\}$ is a controlled Markov chain. We study the convergence under weak assumptions on smoothness-in-$\theta$ of the function $\theta \mapsto H\_{\theta}(x)$. It is usually assumed that this function is continuous for any $x$; in this work, we relax this condition. Our results are illustrated by considering stochastic approximation algorithms for (adaptive) quantile estimation and a penalized version of the vector quantization.