A Gradient Sampling Algorithm for Noisy Nonsmooth Optimization

TL;DR

This paper introduces a gradient sampling algorithm tailored for noisy, nonsmooth, and potentially nonconvex optimization problems, with theoretical guarantees and robust numerical performance.

Contribution

It develops a novel gradient sampling method that handles bounded errors in objective and gradient evaluations, providing probabilistic convergence guarantees.

Findings

The algorithm guarantees either unbounded decrease or stationarity within error bounds.

Numerical experiments demonstrate robust approximate optimization despite noise.

Theoretical analysis confirms convergence properties under bounded errors.

Abstract

An algorithm is proposed, analyzed, and tested for minimizing locally Lipschitz objective functions that may be nonconvex and/or nonsmooth. The algorithm, which is built upon the gradient-sampling methodology, is designed specifically for cases when objective function and generalized gradient values might be subject to bounded uncontrollable errors. Similarly to state-of-the-art guarantees for noisy smooth optimization of this kind, it is proved for the algorithm that, with probability one, either the sequence of objective function values will decrease without bound or the algorithm will generate an iterate at which a measure of stationarity is below a threshold that depends proportionally on the error bounds for the objective function and generalized gradient values. The results of numerical experiments are presented, which show that the algorithm can indeed perform approximate optimization robustly despite errors in objective and generalized gradient values.