Fast gradient descent method for convex optimization problems with an oracle that generates a $(\delta,L)$-model of a function in a requested point

TL;DR

This paper introduces a generalized $(,)$-model of a function that unifies various convex optimization methods, leading to new gradient descent algorithms with broad applicability.

Contribution

It proposes a new $(,)$-model concept that encompasses many existing optimization methods, and develops gradient descent algorithms based on this model.

Findings

Unified framework for convex optimization methods

New gradient descent algorithms with broad applicability

Connections to existing methods like conjugate gradient and proximal methods

Abstract

In this article we propose a new concept of a $(\delta,L)$-model of a function which generalizes the concept of the $(\delta,L)$-oracle (Devolder-Glineur-Nesterov). Using this concept we describe the gradient descent method and the fast gradient descent method and we show that many popular methods (composite optimization methods, level methods, conjugate gradient methods, proximal methods) are special cases of proposed methods in this article.