Adaptation to Inexactness for some Gradient-type Methods

TL;DR

This paper introduces an adaptive gradient method that handles inexact function and gradient evaluations, providing optimal convergence rates for smooth and nonsmooth convex problems with errors, and extends to functions with relaxed Lipschitz conditions.

Contribution

It proposes a novel adaptive gradient method accommodating inexactness in function and gradient evaluations, with proven convergence rates for various convex problem classes.

Findings

Method achieves nearly optimal convergence rates for smooth convex problems with errors.

Extension to nonsmooth convex problems via artificial error introduction.

Linear convergence up to an error-dependent bound.

Abstract

We introduce a notion of inexact model of a convex objective function, which allows for errors both in the function and in its gradient. For this situation, a gradient method with an adaptive adjustment of some parameters of the model is proposed and an estimate for the convergence rate is found. This estimate is optimal on a class of sufficiently smooth problems in the presence of errors. We consider a special class of convex nonsmooth optimization problems. In order to apply the proposed technique to this class, an artificial error should be introduced. We show that the method can be modified for such problems to guarantee a convergence in the function with a nearly optimal rate on the class of convex nonsmooth optimization problems. An adaptive gradient method is proposed for objective functions with some relaxation of the Lipschitz condition for the gradient that satisfy the Polyak--Lojasievicz gradient dominance condition. Here, the objective function and its gradient can be given inexactly. The adaptive choice of the parameters is performed during the operation of the method with respect to both the Lipschitz constant of the gradient and a value corresponding to the error of the gradient and the objective function. The linear convergence of the method is justified up to a value associated with the errors.