Universal gradient descent

TL;DR

This work provides a comprehensive overview of gradient descent, including inexact oracle scenarios, primal-dual properties, and a universal generalization applicable to various optimization methods.

Contribution

It introduces a unified framework for gradient descent that encompasses composite, level's, and proximal methods, and extends to a universal approach.

Findings

Developed a general model for optimized functions including various methods.

Analyzed primal-dual properties within the general model.

Proposed a universal gradient descent method.

Abstract

In this book we collect many different and useful facts around gradient descent method. First of all we consider gradient descent with inexact oracle. We build a general model of optimized function that include composite optimization approach, level's methods, proximal methods etc. Then we investigate primal-dual properties of the gradient descent in general model set-up. At the end we generalize method to universal one.