Global stability of first-order methods for coercive tame functions

TL;DR

This paper proves that certain first-order optimization algorithms for tame, coercive functions tend to stabilize near critical points, ensuring convergence behavior in complex non-smooth settings.

Contribution

It establishes the global stability of various first-order methods for a broad class of tame functions, extending convergence guarantees beyond smooth cases.

Findings

Iterates eventually stay near critical points

Results apply to subgradient, momentum, and coordinate descent methods

Provides convergence insights for non-smooth, tame functions

Abstract

We consider first-order methods with constant step size for minimizing locally Lipschitz coercive functions that are tame in an o-minimal structure on the real field. We prove that if the method is approximated by subgradient trajectories, then the iterates eventually remain in a neighborhood of a connected component of the set of critical points. Under suitable method-dependent regularity assumptions, this result applies to the subgradient method with momentum, the stochastic subgradient method with random reshuffling and momentum, and the random-permutations cyclic coordinate descent method.