A Concise Lyapunov Analysis of Nesterov's Accelerated Gradient Method

TL;DR

This paper presents a straightforward Lyapunov-based proof of the convergence rates for Nesterov's accelerated gradient method applicable to both convex and strongly convex functions, simplifying existing analyses.

Contribution

It offers a concise and direct Lyapunov analysis of Nesterov's method's convergence rates, filling a gap in the simplicity of existing proofs.

Findings

Provides a simple Lyapunov proof for convergence rates

Applies to both convex and strongly convex functions

Enhances understanding of Nesterov's acceleration mechanism

Abstract

Convergence analysis of Nesterov's accelerated gradient method has attracted significant attention over the past decades. While extensive work has explored its theoretical properties and elucidated the intuition behind its acceleration, a simple and direct proof of its convergence rates is still lacking. We provide a concise Lyapunov analysis of the convergence rates of Nesterov's accelerated gradient method for both general convex and strongly convex functions.