Universal Reduced-Operator Method and High-Order Global Curvature Bounds

TL;DR

This paper introduces a universal high-order optimization method that adapts to the complexity of nonlinear operators in metric spaces without requiring smoothness assumptions, achieving optimal convergence rates.

Contribution

The paper proposes the Reduced-Operator Method and a new concept of Global Curvature Bound, enabling universal high-order optimization without smoothness constraints.

Findings

Achieves the fastest universal rate within its class.

Operates without assumptions on smoothness of the operator.

Requires only p-th order derivatives and monotone subproblem solutions.

Abstract

In this paper, we develop a new concept of Global Curvature Bound for an arbitrary nonlinear operator between abstract metric spaces. We use this notion to characterize the global complexity of high-order algorithms solving composite variational problems, which include convex minimization and min-max problems. We develop the new universal Reduced-Operator Method, which automatically achieves the fastest universal rate within our class, while our analysis does not need any specific assumptions about smoothness of the target nonlinear operator. Every step of our universal method of order $p \geq 1$ requires access to the $p$-th order derivative of the operator and the solution of a strictly monotone doubly regularized subproblem. For $p = 1$, this corresponds to computing the standard Jacobian matrix of the operator and solving a simple monotone subproblem, which can be handled using different methods of Convex Optimization.