Rate of Convergence of Truncated Stochastic Approximation Procedures with Moving Bounds

TL;DR

This paper analyzes the convergence and rate of truncated stochastic approximation procedures with moving bounds, random step sizes, and changing regression functions, focusing on parametric statistical estimation.

Contribution

It introduces new convergence rate results for stochastic approximation methods with dynamic truncations and step sizes, extending existing theories.

Findings

Established conditions for convergence with moving bounds

Derived explicit convergence rate bounds

Applied results to parametric statistical estimation

Abstract

The paper is concerned with stochastic approximation procedures having three main characteristics: truncations with random moving bounds, a matrix valued random step-size sequence, and a dynamically changing random regression function. We study convergence and rate of convergence. Main results are supplemented with corollaries to establish various sets of sufficient conditions, with the main emphases on the parametric statistical estimation. The theory is illustrated by examples and special cases.