Revisiting Randomized Smoothing: Nonsmooth Nonconvex Optimization Beyond Global Lipschitz Continuity

TL;DR

This paper extends randomized smoothing techniques to nonsmooth, nonconvex optimization problems without requiring global Lipschitz continuity, introducing new algorithms with provable convergence and improved sample complexity.

Contribution

It introduces a new subgradient growth condition for locally Lipschitz functions and develops randomized smoothing algorithms with convergence guarantees under this milder assumption.

Findings

Algorithms converge to Goldstein stationary points with high probability.

Sample complexity is improved to nearly optimal levels.

Experimental results confirm the effectiveness of the proposed methods.

Abstract

Randomized smoothing is a widely adopted technique for optimizing nonsmooth objective functions. However, its efficiency analysis typically relies on global Lipschitz continuity, a condition rarely met in practical applications. To address this limitation, we introduce a new subgradient growth condition that naturally encompasses a wide range of locally Lipschitz functions, with the classical global Lipschitz function as a special case. Under this milder condition, we prove that randomized smoothing yields a differentiable function that satisfies certain generalized smoothness properties. To optimize such functions, we propose novel randomized smoothing gradient algorithms that, with high probability, converge to $(\delta, \epsilon)$-Goldstein stationary points and achieve a sample complexity of $\tilde{\mathcal{O}}(d^{5/2}\delta^{-1}\epsilon^{-4})$. By incorporating variance reduction techniques, we further improve the sample complexity to $\tilde{\mathcal{O}}(d^{3/2}\delta^{-1}\epsilon^{-3})$, matching the optimal $\epsilon$-bound under the global Lipschitz assumption, up to a logarithmic factor. Experimental results validate the effectiveness of our proposed algorithms.