A geometric alternative to Nesterov's accelerated gradient descent

TL;DR

This paper introduces a new geometric optimization method that matches Nesterov's accelerated gradient descent in convergence rate, offering a simpler interpretation and potential practical improvements.

Contribution

It presents a novel geometric algorithm for smooth strongly convex optimization that achieves optimal convergence rates and offers a new perspective inspired by the ellipsoid method.

Findings

The new method attains the optimal convergence rate of Nesterov's method.

Numerical evidence suggests the new method can outperform Nesterov's in practice.

The algorithm provides a simple geometric interpretation of accelerated optimization.

Abstract

We propose a new method for unconstrained optimization of a smooth and strongly convex function, which attains the optimal rate of convergence of Nesterov's accelerated gradient descent. The new algorithm has a simple geometric interpretation, loosely inspired by the ellipsoid method. We provide some numerical evidence that the new method can be superior to Nesterov's accelerated gradient descent.