Contracting Proximal Methods for Smooth Convex Optimization

TL;DR

This paper introduces contracting proximal methods that accelerate smooth convex optimization by solving contracted subproblems with Bregman divergence, achieving improved convergence rates even with inexact solutions.

Contribution

It presents a unified accelerated framework for smooth convex optimization using contracting proximal steps, applicable to any order of tensor methods.

Findings

Demonstrates acceleration for high-order tensor methods.

Provides global convergence analysis with inexact subproblem solutions.

Achieves near-optimal complexity bounds with limited oracle calls.

Abstract

In this paper, we propose new accelerated methods for smooth convex optimization, called contracting proximal methods. At every step of these methods, we need to minimize a contracted version of the objective function augmented by a regularization term in the form of Bregman divergence. We provide global convergence analysis for a general scheme admitting inexactness in solving the auxiliary subproblem. In the case of using for this purpose high-order tensor methods, we demonstrate an acceleration effect for both convex and uniformly convex composite objective functions. Thus, our construction explains acceleration for methods of any order starting from one. The augmentation of the number of calls of oracle due to computing the contracted proximal steps is limited by the logarithmic factor in the worst-case complexity bound.