Asymptotic normality and optimality in nonsmooth stochastic approximation

TL;DR

This paper extends the theoretical understanding of stochastic approximation algorithms, demonstrating that asymptotic normality and optimality results applicable to smooth problems also hold for certain non-smooth stochastic problems like variational inequalities.

Contribution

It proves that the asymptotic properties and optimality of stochastic approximation methods extend to non-smooth problems such as stochastic variational inequalities.

Findings

Asymptotic normality holds for non-smooth stochastic problems.

Optimality results are valid in non-smooth settings.

Extends classical results to broader classes of stochastic problems.

Abstract

In their seminal work, Polyak and Juditsky showed that stochastic approximation algorithms for solving smooth equations enjoy a central limit theorem. Moreover, it has since been argued that the asymptotic covariance of the method is best possible among any estimation procedure in a local minimax sense of H\'{a}jek and Le Cam. A long-standing open question in this line of work is whether similar guarantees hold for important non-smooth problems, such as stochastic nonlinear programming or stochastic variational inequalities. In this work, we show that this is indeed the case.