Nesterov Finds GRAAL: Optimal and Adaptive Gradient Method for Convex Optimization

TL;DR

This paper introduces an accelerated version of the GRAAL algorithm that adapts to local curvature and achieves near-optimal convergence rates for convex optimization, improving upon previous methods.

Contribution

The authors develop an Nesterov-accelerated GRAAL algorithm that attains optimal convergence rates while maintaining adaptive stepsize estimation without line search.

Findings

Achieves near-optimal iteration complexity for L-smooth functions.

Adapts stepsize to local curvature at a linear rate.

Works under general (L0,L1)-smoothness assumptions.

Abstract

In this paper, we focus on the problem of minimizing a continuously differentiable convex objective function, $\min_x f(x)$. Recently, Malitsky (2020); Alacaoglu et al.(2023) developed an adaptive first-order method, GRAAL. This algorithm computes stepsizes by estimating the local curvature of the objective function without any line search procedures or hyperparameter tuning, and attains the standard iteration complexity $\mathcal{O}(L\lVert x_0-x^*\rVert^2/\epsilon)$ of fixed-stepsize gradient descent for $L$-smooth functions. However, a natural question arises: is it possible to accelerate the convergence of GRAAL to match the optimal complexity $\mathcal{O}(\sqrt{L\lVert x_0-x^*\rVert^2/\epsilon})$ of the accelerated gradient descent of Nesterov (1983)? Although some attempts have been made by Li and Lan (2025); Suh and Ma (2025), the ability of existing accelerated algorithms to adapt to the local curvature of the objective function is highly limited. We resolve this issue and develop GRAAL with Nesterov acceleration, which can adapt its stepsize to the local curvature at a geometric, or linear, rate just like non-accelerated GRAAL. We demonstrate the adaptive capabilities of our algorithm by proving that it achieves near-optimal iteration complexities for $L$-smooth functions, as well as under a more general $(L_0,L_1)$-smoothness assumption (Zhang et al., 2019).