Stochastic Approximation with Discontinuous Dynamics, Differential Inclusions, and Applications

TL;DR

This paper advances stochastic approximation theory by incorporating discontinuous dynamics through differential inclusions and set-valued analysis, providing new convergence results and applications in machine learning and decision processes.

Contribution

It introduces novel convergence results for stochastic approximation algorithms with discontinuities using differential inclusions and stochastic differential inclusions.

Findings

Scaled iterates converge to differential inclusions

Centered scaled iterates converge to stochastic differential inclusions

Applications demonstrated in machine learning algorithms

Abstract

This work develops new results for stochastic approximation algorithms. The emphases are on treating algorithms and limits with discontinuities. The main ingredients include the use of differential inclusions, set-valued analysis, and non-smooth analysis, and stochastic differential inclusions. Under broad conditions, it is shown that a suitably scaled sequence of the iterates has a differential inclusion limit. In addition, it is shown for the first time that a centered and scaled sequence of the iterates converges weakly to a stochastic differential inclusion limit. The results are then used to treat several application examples including Markov decision process, Lasso algorithms, Pegasos algorithms, support vector machine classification, and learning. Some numerical demonstrations are also provided.