On the Convergence Time of Dual Subgradient Methods for Strongly Convex Programs

TL;DR

This paper analyzes the convergence times of dual gradient methods for strongly convex programs, showing improved rates over general convex programs and introducing sliding averages for faster convergence under certain conditions.

Contribution

It establishes faster convergence bounds for dual gradient methods in strongly convex programs and introduces sliding running averages for even quicker convergence when the dual function is locally quadratic.

Findings

Convergence time for strongly convex programs is $O(1/\epsilon)$.

Sliding running averages achieve $O(\log(1/\epsilon))$ convergence under local quadraticity.

Numerical experiments confirm theoretical convergence rates.

Abstract

This paper studies the convergence time of dual gradient methods for general (possibly non-differentiable) strongly convex programs. For general convex programs, the convergence time of dual subgradient/gradient methods with simple running averages (running averages started from iteration $0$) is known to be $O(\frac{1}{\epsilon^{2}})$. This paper shows that the convergence time for general strongly convex programs is $O(\frac{1}{\epsilon})$. This paper also considers a variation of the average scheme, called the sliding running averages, and shows that if the dual function of the strongly convex program is locally quadratic (Note that the locally quadratic property is implied by the locally strongly concave property.) then the convergence time of the dual gradient method with sliding running averages is $O(\log(\frac{1}{\epsilon}))$. The convergence time analysis is further verified by numerical experiments.