Stochastic recursive inclusion in two timescales with an application to the Lagrangian dual problem

TL;DR

This paper introduces a general framework for analyzing two timescale stochastic approximation algorithms, including set-valued mean fields, and applies it to the Lagrangian dual problem in optimization.

Contribution

It extends existing two timescale stochastic approximation frameworks to include set-valued mean fields with easily verifiable assumptions.

Findings

Framework generalizes previous models

Applicable to set-valued mean fields

Analyzes Lagrangian dual problem effectively

Abstract

In this paper we present a framework to analyze the asymptotic behavior of two timescale stochastic approximation algorithms including those with set-valued mean fields. This paper builds on the works of Borkar and Perkins & Leslie. The framework presented herein is more general as compared to the synchronous two timescale framework of Perkins \& Leslie, however the assumptions involved are easily verifiable. As an application, we use this framework to analyze the two timescale stochastic approximation algorithm corresponding to the Lagrangian dual problem in optimization theory.