Adaptive Gradient-type Methods for Convex Optimization Problems with Relative Accuracy and Sharp Minimum

TL;DR

This paper develops and analyzes adaptive gradient methods for convex positively homogeneous optimization problems with relative accuracy, achieving linear convergence and extending to certain non-convex non-smooth problems.

Contribution

It introduces an accelerated universal gradient method and a Polyak stepsize subgradient method with adaptive steps for these problems, including non-convex cases.

Findings

Linear convergence rate for adaptive gradient methods on certain non-smooth problems

Extension of methods to non-convex non-smooth optimization

Effective step adjustment strategies for improved convergence

Abstract

In this paper, we consider gradient-type methods for convex positively homogeneous optimization problems with relative accuracy. An analogue of the accelerated universal gradient-type method for positively homogeneous optimization problems with relative accuracy is investigated. The second approach is related to subgradient methods with B. T. Polyak stepsize. Result on the linear convergence rate for some methods of this type with adaptive step adjustment is obtained for some class of non-smooth problems. Some generalization to a special class of non-convex non-smooth problems is also considered.