Stochastic Subgradient Methods with Guaranteed Global Stability in Nonsmooth Nonconvex Optimization

TL;DR

This paper establishes convergence guarantees and global stability for stochastic subgradient methods in nonsmooth nonconvex optimization, using a Lyapunov-based framework that handles a broad class of algorithms.

Contribution

It introduces a unified framework for analyzing the global stability of stochastic subgradient methods, including cases with non-coercive Lyapunov functions, advancing theoretical understanding.

Findings

Proves uniform boundedness and asymptotic stability of iterates.

Develops a framework applicable to a wide range of stochastic subgradient methods.

Shows potential for broad applicability in nonsmooth nonconvex optimization.

Abstract

In this paper, we focus on providing convergence guarantees for stochastic subgradient methods in minimizing nonsmooth nonconvex functions. We first investigate the global stability of a general framework for stochastic subgradient methods, where the corresponding differential inclusion admits a coercive Lyapunov function. We prove that, for any sequence of sufficiently small stepsizes and approximation parameters, coupled with sufficiently controlled noises, the iterates are uniformly bounded and asymptotically stabilize around the stable set of its corresponding differential inclusion. Moreover, we develop an improved analysis to apply our proposed framework to establish the global stability of a wide range of stochastic subgradient methods, where the corresponding Lyapunov functions are possibly non-coercive. These theoretical results illustrate the promising potential of our proposed framework for establishing the global stability of various stochastic subgradient methods.