The cost of nonconvexity in deterministic nonsmooth optimization

TL;DR

This paper analyzes how nonconvexity affects the complexity of nonsmooth optimization, providing new bounds for deterministic algorithms on various classes of functions, including piecewise linear and difference-of-convex objectives.

Contribution

It introduces a simple deterministic black-box algorithm with complexity bounds that depend on a natural nonconvexity measure, extending previous results to broader nonconvex settings.

Findings

Complexity bounds of O(ε^{-5}) for difference-of-convex objectives.

Complexity bounds of O(ε^{-4}) for weakly convex functions.

Dependence of complexity on a nonconvexity modulus related to second derivatives.

Abstract

We study the impact of nonconvexity on the complexity of nonsmooth optimization, emphasizing objectives such as piecewise linear functions, which may not be weakly convex. We focus on a dimension-independent analysis, slightly modifying a black-box algorithm of Zhang et al. (2020) that approximates an $\epsilon$-stationary point of any directionally differentiable Lipschitz objective using $O(\epsilon^{-4})$ calls to a specialized subgradient oracle and a randomized line search. Our simple black-box deterministic version, achieves $O(\epsilon^{-5})$ for any difference-of-convex objective, and $O(\epsilon^{-4})$ for the weakly convex case. Our complexity bound depends on a natural nonconvexity modulus, related, intriguingly, to the negative part of directional second derivatives of the objective, understood in the distributional sense.