The Method of Ellipcenters for strongly convex minimization

TL;DR

This paper analyzes the Method of Ellipcenters (ME), an accelerated first-order scheme for strongly convex minimization, demonstrating its convergence for differentiable objectives and showcasing its superior numerical performance.

Contribution

The paper extends the convergence analysis of ME to all differentiable strongly convex functions and highlights its promising numerical advantages over existing methods.

Findings

ME converges linearly for quadratic strongly convex functions.

Numerical experiments show ME outperforms steepest descent, FISTA, Barzilai-Borwein, and Conjugate Gradient.

Theoretical results suggest potential for broader applications.

Abstract

This work is about ME, the Method of Ellipcenters. ME was recently introduced by these very authors as a first order accelerated scheme for unconstrained minimization. Its iterates are all centers of ellipses carefully designed to somehow capture ill-conditioning of the underlying optimization problem. In the first article on ME, we were able to prove that it converges with linear rate when the objective function is quadratic and strongly convex, while here we derive convergence for any differentiable strongly convex objective. This investigation was inspired by the great performance of ME in quadratic minimization against steepest descent with exact line search, FISTA, Barzilai-Borwein and Conjugate Gradient. The experiments we carry out now, make ME even more attractive from the numerical point of view. On top of that, the theory seems promising for quite more general settings.