Central Limit Theorems for Stochastic Approximation with controlled Markov chain dynamics

TL;DR

This paper establishes a Central Limit Theorem for stochastic approximation processes driven by controlled Markov chain dynamics, including multiple targets and truncated algorithms, extending previous theoretical results.

Contribution

It introduces CLT results for SA with controlled Markov chain dynamics and multiple targets, improving upon existing theoretical frameworks.

Findings

Provides sufficient conditions for CLT validity in controlled Markov settings

Extends CLT applicability to multiple target scenarios

Includes analysis of truncated stochastic approximation algorithms

Abstract

This paper provides a Central Limit Theorem (CLT) for a process $\{\theta_n, n\geq 0\}$ satisfying a stochastic approximation (SA) equation of the form $\theta_{n+1} = \theta_n + \gamma_{n+1} H(\theta_n,X_{n+1})$; a CLT for the associated average sequence is also established. The originality of this paper is to address the case of controlled Markov chain dynamics $\{X_n, n\geq 0 \}$ and the case of multiple targets. The framework also accomodates (randomly) truncated SA algorithms. Sufficient conditions for CLT's to hold are provided as well as comments on how these conditions extend previous works (such as independent and identically distributed dynamics, the Robbins-Monro dynamic or the single target case). The paper gives a special emphasis on how these conditions hold for SA with controlled Markov chain dynamics and multiple targets; it is proved that this paper improves on existing works.