Inexact Tensor Methods with Dynamic Accuracies

TL;DR

This paper introduces inexact high-order tensor methods with dynamic accuracy strategies for convex optimization, achieving optimal convergence rates and demonstrating effectiveness through computational experiments.

Contribution

It proposes two novel dynamic accuracy strategies for inexact tensor methods, maintaining convergence rates and enabling accelerated schemes with superlinear local convergence.

Findings

Methods achieve the same global convergence rate as error-free versions

The second accuracy approach yields local superlinear convergence for p ≥ 2

Computational results validate the effectiveness across machine learning problems

Abstract

In this paper, we study inexact high-order Tensor Methods for solving convex optimization problems with composite objective. At every step of such methods, we use approximate solution of the auxiliary problem, defined by the bound for the residual in function value. We propose two dynamic strategies for choosing the inner accuracy: the first one is decreasing as $1/k^{p + 1}$, where $p \geq 1$ is the order of the method and $k$ is the iteration counter, and the second approach is using for the inner accuracy the last progress in the target objective. We show that inexact Tensor Methods with these strategies achieve the same global convergence rate as in the error-free case. For the second approach we also establish local superlinear rates (for $p \geq 2$), and propose the accelerated scheme. Lastly, we present computational results on a variety of machine learning problems for several methods and different accuracy policies.