Escaping strict saddle points of the Moreau envelope in nonsmooth optimization

TL;DR

This paper extends stochastic gradient methods to nonsmooth optimization by analyzing how they can escape strict saddle points of the Moreau envelope, providing theoretical guarantees for a broad class of algorithms.

Contribution

It introduces an analysis of inexact stochastic gradient methods applied to the Moreau envelope, demonstrating their ability to escape saddle points in nonsmooth optimization.

Findings

Algorithms can escape strict saddle points at a controlled rate

Proximal subproblem directions approximate the Moreau envelope gradient

Theoretical guarantees extend stochastic methods to nonsmooth settings

Abstract

Recent work has shown that stochastically perturbed gradient methods can efficiently escape strict saddle points of smooth functions. We extend this body of work to nonsmooth optimization, by analyzing an inexact analogue of a stochastically perturbed gradient method applied to the Moreau envelope. The main conclusion is that a variety of algorithms for nonsmooth optimization can escape strict saddle points of the Moreau envelope at a controlled rate. The main technical insight is that typical algorithms applied to the proximal subproblem yield directions that approximate the gradient of the Moreau envelope in relative terms.