Proximally Guided Stochastic Subgradient Method for Nonsmooth, Nonconvex Problems

TL;DR

This paper proposes a stochastic subgradient method for weakly convex nonsmooth nonconvex functions, providing the first convergence rate analysis for such functions, and demonstrating its effectiveness similar to smooth nonconvex cases.

Contribution

Introduces a stochastic projected subgradient method with convergence analysis for weakly convex functions, a broad class including additive and convex composite functions.

Findings

Converges at the same rate as stochastic gradient methods for smooth nonconvex problems.

First convergence rate analysis for stochastic subgradient methods on weakly convex functions.

Applicable to a wide class of nonsmooth, nonconvex functions.

Abstract

In this paper, we introduce a stochastic projected subgradient method for weakly convex (i.e., uniformly prox-regular) nonsmooth, nonconvex functions---a wide class of functions which includes the additive and convex composite classes. At a high-level, the method is an inexact proximal point iteration in which the strongly convex proximal subproblems are quickly solved with a specialized stochastic projected subgradient method. The primary contribution of this paper is a simple proof that the proposed algorithm converges at the same rate as the stochastic gradient method for smooth nonconvex problems. This result appears to be the first convergence rate analysis of a stochastic (or even deterministic) subgradient method for the class of weakly convex functions.