Convergence of the Gradient Sampling Algorithm on Directionally Lipschitz Functions

TL;DR

This paper extends the convergence theory of the gradient sampling algorithm to a broader class of functions called directionally Lipschitz, which are not necessarily locally Lipschitz but are almost everywhere differentiable.

Contribution

The paper develops a convergence analysis for the gradient sampling algorithm applied to directionally Lipschitz functions, addressing challenges posed by their Clarke subdifferential properties.

Findings

Convergence theory is extended to directionally Lipschitz functions.

The analysis recovers standard results for locally Lipschitz functions.

Iterates converge to points with either empty Clarke subdifferential or degenerate steepest descent.

Abstract

The convergence theory for the gradient sampling algorithm is extended to directionally Lipschitz functions. Although directionally Lipschitz functions are not necessarily locally Lipschitz, they are almost everywhere differentiable and well approximated by gradients and so are a natural candidate for the application of the gradient sampling algorithm. The main obstacle to this extension is the potential unboundedness or emptiness of the Clarke subdifferential at points of interest. The convergence analysis we present provides one path to addressing these issues. In particular, we recover the usual convergence theory when the function is locally Lipschitz. Moreover, if the algorithm does not drive a certain measure of criticality to zero, then the iterates must converge to a point at which either the Clarke subdifferential is empty or the direction of steepest descent is degenerate in the sense that it does lie in the interior of the domain of the regular subderivative.