Lp and almost sure rates of convergence of averaged stochastic gradient algorithms: locally strongly convex objective

TL;DR

This paper analyzes the convergence rates of averaged stochastic gradient algorithms for minimizing locally strongly convex functions, providing both asymptotic and non-asymptotic results relevant for large-scale, high-dimensional data settings.

Contribution

It offers new theoretical insights into the convergence behavior of stochastic gradient algorithms under local strong convexity assumptions.

Findings

Derived asymptotic convergence rates for stochastic gradient estimates.

Established non-asymptotic convergence bounds for averaged algorithms.

Applicable to high-dimensional, large-sample statistical problems.

Abstract

An usual problem in statistics consists in estimating the minimizer of a convex function. When we have to deal with large samples taking values in high dimensional spaces, stochastic gradient algorithms and their averaged versions are efficient candidates. Indeed, (1) they do not need too much computational efforts, (2) they do not need to store all the data, which is crucial when we deal with big data, (3) they allow to simply update the estimates, which is important when data arrive sequentially. The aim of this work is to give asymptotic and non asymptotic rates of convergence of stochastic gradient estimates as well as of their averaged versions when the function we would like to minimize is only locally strongly convex.