Adaptive Accelerated Gradient Method for Smooth Convex Optimization

TL;DR

This paper introduces an adaptive accelerated gradient method for smooth convex optimization that automatically estimates the smoothness constant, achieving fast convergence rates without line search.

Contribution

It presents a novel adaptive scheme for accelerated gradient methods that does not require line search and adapts to unknown smoothness constants.

Findings

Converges at a rate of O(1/k^2) for general smooth convex functions.

Achieves linear convergence when the function is strongly convex.

Does not rely on line search procedures for step size determination.

Abstract

We propose an adaptive accelerated gradient method for solving smooth convex optimization problems. The method incorporates a scheme to determine the step size adaptively, by means of a local estimation of the smoothness constant, which is assumed unknown, without resorting to line search procedures. The sequence generated by this method converges weakly to a minimizer of the objective function, and the function values converge at a fast rate of $\mathcal{O}\left( \frac{1}{k^2} \right)$. Moreover, if the objective function is strongly convex, the function values converge at a linear rate.