Subgradient Gliding Method for Nonsmooth Convex Optimization

TL;DR

This paper introduces the subgradient gliding method, a novel approach for nonsmooth convex optimization that overcomes the limitations of the classical subgradient method, ensuring reliable convergence even at boundary points without Lipschitz continuity.

Contribution

The paper proposes the subgradient gliding method, extending classical subgradient techniques to handle boundary points and non-Lipschitz functions, with proven optimal convergence rates and broad applicability.

Findings

The method reliably converges where classical methods fail.

Achieves optimal ergodic convergence rates for convex and strongly convex problems.

Demonstrates significant improvements in accuracy and speed in numerical experiments.

Abstract

We identify and analyze a fundamental limitation of the classical projected subgradient method in nonsmooth convex optimization: the inevitable failure caused by the absence of valid subgradients at boundary points. We show that, under standard step sizes for both convex and strongly convex objectives, the method can fail after a single iteration with probability arbitrarily close to one, even on simple problem instances. To overcome this limitation, we propose a novel alternative termed the \textit{subgradient gliding method}, which remains well defined without boundary subgradients and avoids premature termination. Beyond resolving this foundational issue, the proposed framework encompasses the classical projected subgradient method as a special case and substantially enlarges its admissible step-size design space, providing greater flexibility for algorithmic design. We establish optimal ergodic convergence rates, $\mathcal{O}(1/\sqrt{t})$ for convex problems and $\mathcal{O}(1/t)$ for strongly convex problems, and further extend the framework to stochastic settings. Notably, our analysis does not rely on global Lipschitz continuity of the objective function, requiring only mild control on subgradient growth. Numerical experiments demonstrate that, in scenarios where the classical projected subgradient method fails completely, the proposed method converges reliably with a $100\%$ success rate and achieves orders-of-magnitude improvements in accuracy and convergence speed. These results substantially expand the scope of subgradient-based optimization methods to non-Lipschitz nonsmooth convex problems.