On the Hardness of Meaningful Local Guarantees in Nonsmooth Nonconvex Optimization

TL;DR

This paper demonstrates that, unlike smooth optimization, nonsmooth nonconvex problems with local information are inherently hard to solve efficiently, as no algorithms can reliably find local minima in sub-exponential time in the worst case.

Contribution

It proves an unconditional hardness result for local algorithms in nonsmooth nonconvex optimization, showing they cannot achieve meaningful guarantees in sub-exponential time.

Findings

Local algorithms cannot guarantee function value improvements efficiently in worst-case nonsmooth nonconvex problems.

Contrasts with smooth optimization where gradient methods are dimension-independent and efficient.

Hardness holds unconditionally, not relying on complexity conjectures.

Abstract

We study the oracle complexity of nonsmooth nonconvex optimization, with the algorithm assumed to have access only to local function information. It has been shown by Davis, Drusvyatskiy, and Jiang (2023) that for nonsmooth Lipschitz functions satisfying certain regularity and strictness conditions, perturbed gradient descent converges to local minimizers asymptotically. Motivated by this result and by other recent algorithmic advances in nonconvex nonsmooth optimization concerning Goldstein stationarity, we consider the question of obtaining a non-asymptotic rate of convergence to local minima for this problem class. We provide the following negative answer to this question: Local algorithms acting on regular Lipschitz functions cannot, in the worst case, provide meaningful local guarantees in terms of function value in sub-exponential time, even when all near-stationary points are global minima. This sharply contrasts with the smooth setting, for which it is well-known that standard gradient methods can do so in a dimension-independent rate. Our result complements the rich body of work in the theoretical computer science literature that provide hardness results conditional on conjectures such as $\mathsf{P}\neq\mathsf{NP}$ or cryptographic assumptions, in that ours holds unconditional of any such assumptions.