On convergence rates of subgradient descent on semialgebraic functions

TL;DR

This paper establishes convergence rates for the subgradient method on semialgebraic functions by leveraging geometric stratifications and curvature bounds, extending classical results to nonsmooth, nonconvex settings.

Contribution

It introduces a geometric framework using stratifications and curvature bounds to analyze subgradient descent on semialgebraic functions, providing explicit convergence rates.

Findings

Convergence rates depend on the stratification dimension and improve with fewer strata.

The framework applies to functions definable in polynomially bounded o-minimal structures.

It extends to decreasing step sizes and recovers known results in the smooth case.

Abstract

We analyze the constant step size subgradient method on nonsmooth, nonconvex functions. We identify geometric assumptions on the objective function under which i) its domain admits a partition (stratification) into smooth manifolds (strata) on which the function is smooth; ii) a global projection formula for Clarke subgradients holds; and iii) quantitative curvature bounds hold on each stratum. Under these conditions, we prove that the iterates of the subgradient method locally shadow a Riemannian gradient descent on nearby strata, which we use to measure stationarity. We introduce a selection rule for the active stratum and develop a mechanism that assembles local descent inequalities across successive strata into explicit convergence rates. These rates are expressed in terms of the number of dimensions present in the stratification, improve as the number of strata decreases, and recover, up to constants, the classical rates in the smooth case. We show that the stated assumptions follow from the existence of Lipschitz stratifications of semialgebraic sets, and are therefore automatically satisfied for semialgebraic functions and, more generally, for functions definable in polynomially bounded o-minimal structures, yielding the first known convergence rates in these settings. As intermediate results of independent interest, we establish tubular neighborhood estimates for Lipschitz stratifications and a global projection formula for Clarke subgradients. Finally, we show that our framework extends to decreasing step size and recovers, via an alternative argument, the recently announced result of Lai and Song on sequential convergence of the subgradient method with step sizes 1/k.