An Optimal Affine Invariant Smooth Minimization Algorithm

TL;DR

This paper develops an affine invariant accelerated optimization algorithm that adapts to problem geometry and smoothness, providing optimal complexity bounds for a broad class of convex problems.

Contribution

It introduces an affine invariant implementation of Nesterov's accelerated method, extending it to H"older smooth functions with adaptive complexity bounds.

Findings

Complexity bounds are proportional to an affine invariant regularity constant.

The algorithm adapts to H"older smoothness and gradient Lipschitz constants.

Matching lower bounds are established for $\, ext{ell}_p$ ball feasible sets.

Abstract

We formulate an affine invariant implementation of the accelerated first-order algorithm in Nesterov (1983). Its complexity bound is proportional to an affine invariant regularity constant defined with respect to the Minkowski gauge of the feasible set. We extend these results to more general problems, optimizing H\"older smooth functions using $p$-uniformly convex prox terms, and derive an algorithm whose complexity better fits the geometry of the feasible set and adapts to both the best H\"older smoothness parameter and the best gradient Lipschitz constant. Finally, we detail matching complexity lower bounds when the feasible set is an $\ell_p$ ball. In this setting, our upper bounds on iteration complexity for the algorithm in Nesterov (1983) are thus optimal in terms of target precision, smoothness and problem dimension.