Nonmonotone subgradient methods based on a local descent lemma

TL;DR

This paper introduces a nonmonotone line search subgradient algorithm for upper-$\mathcal{C}^2$ functions, proving convergence and demonstrating advantages over existing methods, especially in clustering tasks.

Contribution

The paper develops a novel nonmonotone subgradient algorithm with an adaptive scheme, applicable to nonsmooth, nonconvex functions, and provides practical implementation for clustering.

Findings

Proves subsequential convergence to stationary points.

Demonstrates the effectiveness of SNSM in clustering problems.

Shows advantages of SNSM over existing algorithms in numerical experiments.

Abstract

In this paper we present a nonmonotone line search subgradient algorithm tailored to upper-$\mathcal{C}^2$ functions. This is a family of nonsmooth and nonconvex functions that satisfies a nonsmooth and local version of the descent lemma, making them suitable for line searches. We prove subsequential convergence of the proposed algorithm to a stationary point of the optimization problem. Our approach allows us to cover the setting of various subgradient algorithms, including Newton and quasi-Newton methods. In addition, we propose a specification of the general scheme, named Self-adaptive Nonmonotone Subgradient Method (SNSM), which automatically updates the parameters of the line search. Particular attention is paid to the minimum sum-of-squares clustering problem, for which we provide a concrete implementation of SNSM. We conclude with some numerical experiments where we exhibit the advantages of SNSM in comparison with some known algorithms.