Gradient Sampling Methods with Inexact Subproblem Solutions and Gradient Aggregation

TL;DR

This paper introduces inexact subproblem solutions and gradient aggregation strategies into gradient sampling methods, maintaining convergence guarantees while significantly reducing computational effort in nonsmooth, nonconvex optimization.

Contribution

It proposes novel inexact solution techniques and gradient aggregation methods for GS, with proven convergence and improved computational efficiency.

Findings

Inexact subproblem solutions reduce computational effort.

Gradient aggregation significantly decreases overall computation.

Theoretical convergence guarantees are preserved with new strategies.

Abstract

Gradient sampling (GS) has proved to be an effective methodology for the minimization of objective functions that may be nonconvex and/or nonsmooth. The most computationally expensive component of a contemporary GS method is the need to solve a convex quadratic subproblem in each iteration. In this paper, a strategy is proposed that allows the use of inexact solutions of these subproblems, which, as proved in the paper, can be incorporated without the loss of theoretical convergence guarantees. Numerical experiments show that by exploiting inexact subproblem solutions, one can consistently reduce the computational effort required by a GS method. Additionally, a strategy is proposed for aggregating gradient information after a subproblem is solved (potentially inexactly), as has been exploited in bundle methods for nonsmooth optimization. It is proved that the aggregation scheme can be introduced without the loss of theoretical convergence guarantees. Numerical experiments show that incorporating this gradient aggregation approach substantially reduces the computational effort required by a GS method.