Adaptive Gradient Methods for Some Classes of Non-Smooth Optimization Problems

TL;DR

This paper introduces adaptive gradient algorithms for non-smooth convex optimization, utilizing inexact models to improve convergence and applicability to variational inequalities and saddle point problems.

Contribution

It presents novel adaptive methods based on inexact ($ \delta, \Delta, \\ L$)-models for various non-smooth optimization problems, extending existing techniques.

Findings

Developed inexact ($ \\delta, \\Delta, \\ L$)-model framework

Proposed adaptive gradient methods for non-smooth convex problems

Extended methods to variational inequalities and saddle point problems

Abstract

We propose several adaptive algorithmic methods for problems of non-smooth convex optimization. The first of them is based on a special artificial inexactness. Namely, the concept of inexact ($ \delta, \Delta, L$)-model of objective functional in optimization is introduced and some gradient-type methods with adaptation of inexactness parameters are proposed. A similar concept of an inexact model is introduced for variational inequalities as well as for saddle point problems. Analogues of switching sub-gradient schemes are proposed for convex programming problems with some general assumptions.