Subgradient methods for non-smooth optimization problems with some relaxation of sharp minimum

TL;DR

This paper extends subgradient methods for non-smooth optimization by introducing a generalized sharp minimum condition, enabling effective use with inexact information and broader problem classes, including quasi-convexity.

Contribution

It proposes a generalized sharp minimum condition, allowing subgradient methods to work with inexact data and wider classes of non-smooth problems, including quasi-convex and strongly convex cases.

Findings

Applicable to strongly convex non-smooth problems

Experimental comparison with optimal subgradient methods

Extended to problems with convexity relaxations like quasi-convexity

Abstract

In this paper we propose a generalized condition for a sharp minimum, somewhat similar to the inexact oracle proposed recently by Devolder-Glineur-Nesterov. The proposed approach makes it possible to extend the class of applicability of subgradient methods with the Polyak step-size, to the situation of inexact information about the value of the minimum, as well as the unknown Lipschitz constant of the objective function. Moreover, the use of local analogs of the global characteristics of the objective function makes it possible to apply the results of this type to wider classes of problems. We show the possibility of applying the proposed approach to strongly convex non-smooth problems, also, we make an experimental comparison with the known optimal subgradient method for such a class of problems. Moreover, there were obtained some results connected to the applicability of the proposed technique to some types of problems with convexity relaxations: the recently proposed notion of weak $\beta$-quasi-convexity and ordinary quasi-convexity. Also in the paper, we study a generalization of the described technique to the situation with the assumption that the $\delta$-subgradient of the objective function is available instead of the usual subgradient. For one of the considered methods, conditions are found under which, in practice, it is possible to escape the projection of the considered iterative sequence onto the feasible set of the problem.