Stochastic subgradient method converges at the rate $O(k^{-1/4})$ on weakly convex functions

TL;DR

This paper proves that the proximal stochastic subgradient method converges at a rate of $O(k^{-1/4})$ for weakly convex functions, resolving an open question about its efficiency in nonconvex optimization.

Contribution

It establishes the convergence rate of the proximal stochastic subgradient method on weakly convex functions, addressing an open problem in nonconvex optimization.

Findings

Convergence rate of $O(k^{-1/4})$ for the method.

Resolution of an open question on stochastic gradient methods.

Applicability to minimizing sums of smooth nonconvex and convex functions.

Abstract

We prove that the proximal stochastic subgradient method, applied to a weakly convex problem, drives the gradient of the Moreau envelope to zero at the rate $O(k^{-1/4})$. As a consequence, we resolve an open question on the convergence rate of the proximal stochastic gradient method for minimizing the sum of a smooth nonconvex function and a convex proximable function.