Inexact Tensor Methods and Their Application to Stochastic Convex Optimization

TL;DR

This paper introduces inexact tensor methods for stochastic convex optimization, analyzing their convergence and providing conditions for inexact derivatives to achieve desired accuracy, including stochastic variants with mini-batch size guidelines.

Contribution

It develops general inexact tensor algorithms with convergence analysis and conditions, extending to stochastic settings with practical mini-batch size recommendations.

Findings

Convergence rates for inexact tensor methods are established.

Conditions for derivative inexactness ensuring accuracy are provided.

Sufficient mini-batch sizes for stochastic tensor methods are derived.

Abstract

We propose general non-accelerated and accelerated tensor methods under inexact information on the derivatives of the objective, analyze their convergence rate. Further, we provide conditions for the inexactness in each derivative that is sufficient for each algorithm to achieve the desired accuracy. As a corollary, we propose stochastic tensor methods for convex optimization and obtain sufficient mini-batch sizes for each derivative.