An Explicit Convergence Rate for Nesterov's Method from SDP

TL;DR

This paper derives an explicit analytical upper bound on the convergence rate of Nesterov's accelerated method for strongly convex functions using SDP and IQC techniques, improving upon previous bounds.

Contribution

It provides the first explicit analytical solution to the SDP for Nesterov's method, yielding a new, optimized convergence rate bound.

Findings

Explicit convergence rate bound derived

Bound is the best known for strongly convex functions

Optimization over parameters improves the bound

Abstract

The framework of Integral Quadratic Constraints (IQC) introduced by Lessard et al. (2014) reduces the computation of upper bounds on the convergence rate of several optimization algorithms to semi-definite programming (SDP). In particular, this technique was applied to Nesterov's accelerated method (NAM). For quadratic functions, this SDP was explicitly solved leading to a new bound on the convergence rate of NAM, and for arbitrary strongly convex functions it was shown numerically that IQC can improve bounds from Nesterov (2004). Unfortunately, an explicit analytic solution to the SDP was not provided. In this paper, we provide such an analytical solution, obtaining a new general and explicit upper bound on the convergence rate of NAM, which we further optimize over its parameters. To the best of our knowledge, this is the best, and explicit, upper bound on the convergence rate of NAM for strongly convex functions.