Constrained and Preconditioned Stochastic Gradient Method

TL;DR

This paper introduces a constrained, preconditioned stochastic gradient method (PSGM) for applications like data communication and image processing, demonstrating its convergence and effectiveness through analysis and simulations.

Contribution

It proposes a novel PSGM that incorporates constraints and preconditioning, with convergence guarantees under certain conditions.

Findings

PSGM converges to the best approximation under specified assumptions.

Simulation results confirm the effectiveness of PSGM.

Constraints improve stochastic approximation performance.

Abstract

We consider stochastic approximations which arise from such applications as data communications and image processing. We demonstrate why constraints are needed in a stochastic approximation and how a constrained approximation can be incorporated into a preconditioning technique to derive the pre-conditioned stochastic gradient method (PSGM). We perform convergence analysis to show that the PSGM converges to the theoretical best approximation under some simple assumptions on the preconditioner and on the independence of samples drawn from a stochastic process. Simulation results are presented to demonstrate the effectiveness of the constrained and precondi-tioned stochastic gradient method.