Unifying restart accelerated gradient and proximal bundle methods

TL;DR

This paper introduces a unified framework that connects restart accelerated gradient methods and proximal bundle methods, providing optimal convergence results and new insights into their algorithmic principles for convex optimization.

Contribution

It unifies two prominent optimization methods within a common inexact proximal point framework, revealing their underlying algorithmic connection and optimal iteration complexity.

Findings

The restart accelerated gradient method achieves optimal iteration complexity.

Proximal bundle method is shown to be an instance of the inexact proximal point framework.

Provides new insights into the algorithmic principles linking different convex optimization methods.

Abstract

This paper presents a novel restarted version of Nesterov's accelerated gradient method and establishes its optimal iteration-complexity for solving convex smooth composite optimization problems. The proposed restart accelerated gradient method is shown to be a specific instance of the accelerated inexact proximal point framework introduced in "An accelerated hybrid proximal extragradient method for convex optimization and its implications to second-order methods" by Monteiro and Svaiter, SIAM Journal on Optimization, 2013. Furthermore, this work examines the proximal bundle method within the inexact proximal point framework, demonstrating that it is an instance of the framework. Notably, this paper provides new insights into the underlying algorithmic principle that unifies two seemingly disparate optimization methods, namely, the restart accelerated gradient and the proximal bundle methods.