Truncated Stochastic Approximation with Moving Bounds: Convergence

TL;DR

This paper introduces a broad class of truncated stochastic approximation methods with moving bounds, designed for statistical estimation, featuring random bounds, matrix step-sizes, and dynamic regression functions, with proven convergence.

Contribution

The paper develops a new class of stochastic approximation procedures with moving bounds and matrix step-sizes, expanding their applicability to statistical estimation problems.

Findings

Proved convergence of the proposed procedures.

Illustrated the methods with several examples.

Enhanced stochastic approximation techniques for statistical estimation.

Abstract

In this paper we propose a wide class of truncated stochastic approximation procedures with moving random bounds. While we believe that the proposed class of procedures will find its way to a wider range of applications, the main motivation is to accommodate applications to parametric statistical estimation theory. Our class of stochastic approximation procedures has three main characteristics: truncations with random moving bounds, a matrix valued random step-size sequence, and dynamically changing random regression function. We establish convergence and consider several examples to illustrate the results.