Asymptotic Behaviour of Truncated Stochastic Approximation Procedures

TL;DR

This paper analyzes the long-term behavior of advanced stochastic approximation algorithms with random truncations, matrix step-sizes, and dynamic functions, establishing their asymptotic linearity and distributional properties.

Contribution

It introduces a framework for understanding the asymptotic behavior of stochastic approximation procedures with complex, random components, extending existing theory.

Findings

Procedures are asymptotically linear under mild conditions

Central limit theorem applies to these procedures

Examples illustrate the theoretical results

Abstract

We study asymptotic behaviour of stochastic approximation procedures with three main characteristics: truncations with random moving bounds, a matrix valued random step-size sequence, and a dynamically changing random regression function. In particular, we show that under quite mild conditions, stochastic approximation procedures are asymptotically linear in the statistical sense, that is, they can be represented as weighted sums of random variables. Therefore, a suitable form of the central limit theorem can be applied to derive asymptotic distribution of the corresponding processes. The theory is illustrated by various examples and special cases.