Variants of the A-HPE and large-step A-HPE algorithms for strongly convex problems with applications to accelerated high-order tensor methods

TL;DR

This paper introduces variants of the A-HPE and large-step A-HPE algorithms with proven linear and superlinear convergence rates for strongly convex problems, and applies these to develop new high-order tensor methods with improved iteration complexity.

Contribution

It proposes new variants of A-HPE algorithms with enhanced convergence rates and applies them to high-order tensor methods for strongly convex optimization.

Findings

Proven linear and superlinear convergence rates for the new algorithms.

Developed a new inexact tensor method with optimal iteration complexity.

Achieved a fast global convergence rate for the inexact Newton-proximal algorithm.

Abstract

For solving strongly convex optimization problems, we propose and study the global convergence of variants of the A-HPE and large-step A-HPE algorithms of Monteiro and Svaiter. We prove linear and the superlinear $\mathcal{O}\left(k^{\,-k\left(\frac{p-1}{p+1}\right)}\right)$ global rates for the proposed variants of the A-HPE and large-step A-HPE methods, respectively. The parameter $p\geq 2$ appears in the (high-order) large-step condition of the new large-step A-HPE algorithm. We apply our results to high-order tensor methods, obtaning a new inexact (relative-error) tensor method for (smooth) strongly convex optimization with iteration-complexity $\mathcal{O}\left(k^{\,-k\left(\frac{p-1}{p+1}\right)}\right)$. In particular, for $p=2$, we obtain an inexact Newton-proximal algorithm with fast global $\mathcal{O}\left(k^{\,-k/3}\right)$ convergence rate.