Goldstein Stationarity in Lipschitz Constrained Optimization

TL;DR

This paper establishes the first convergence guarantees for a subgradient method in Lipschitz constrained optimization without smoothness or convexity assumptions, extending recent unconstrained minimization techniques to constrained settings.

Contribution

It generalizes Lipschitz unconstrained minimization techniques to handle inequality constraints, providing convergence guarantees for a subgradient method to Goldstein stationary points.

Findings

Proves convergence guarantees for subgradient methods in Lipschitz constrained optimization.

Extends recent unconstrained minimization techniques to constrained problems.

Provides guarantees on reaching Goldstein Fritz-John or KKT stationary points.

Abstract

We prove the first convergence guarantees for a subgradient method minimizing a generic Lipschitz function over generic Lipschitz inequality constraints. No smoothness or convexity (or weak convexity) assumptions are made. Instead, we utilize a sequence of recent advances in Lipschitz unconstrained minimization, which showed convergence rates of $O(1/\delta\epsilon^3)$ towards reaching a "Goldstein" stationary point, that is, a point where an average of gradients sampled at most distance $\delta$ away has size at most $\epsilon$. We generalize these prior techniques to handle functional constraints, proposing a subgradient-type method with similar $O(1/\delta\epsilon^3)$ guarantees on reaching a Goldstein Fritz-John or Goldstein KKT stationary point, depending on whether a certain Goldstein-style generalization of constraint qualification holds.