Accelerated gradient methods with absolute and relative noise in the gradient

TL;DR

This paper analyzes accelerated gradient methods for smooth convex optimization under inexact gradient information with absolute and relative noise, providing error bounds, stopping criteria, and extensions to stochastic and nonsmooth problems.

Contribution

It introduces error analysis for accelerated methods with unbounded feasible sets under absolute and relative gradient noise, including stopping criteria and extensions.

Findings

Error bounds for convex and strongly convex cases

Trajectory bounds for unbounded feasible sets

Extensions to stochastic and nonsmooth optimization

Abstract

In this paper, we investigate accelerated first-order methods for smooth convex optimization problems under inexact information on the gradient of the objective. The noise in the gradient is considered to be additive with two possibilities: absolute noise bounded by a constant, and relative noise proportional to the norm of the gradient. We investigate the accumulation of the errors in the convex and strongly convex settings with the main difference with most of the previous works being that the feasible set can be unbounded. The key to the latter is to prove a bound on the trajectory of the algorithm. We also give a stopping criterion for the algorithm and consider extensions to the cases of stochastic optimization and composite nonsmooth problems.