Convergence rates of proximal gradient methods via the convex conjugate

TL;DR

This paper presents a new proof technique for the convergence rates of proximal gradient methods in convex optimization, using convex conjugates to establish bounds.

Contribution

It introduces a novel proof approach leveraging convex conjugates to derive convergence rates for proximal gradient algorithms.

Findings

Proximal gradient methods achieve $O(1/k)$ convergence.

Accelerated proximal gradient methods achieve $O(1/k^2)$ convergence.

The proof simplifies understanding of convergence behavior.

Abstract

We give a novel proof of the $O(1/k)$ and $O(1/k^2)$ convergence rates of the proximal gradient and accelerated proximal gradient methods for composite convex minimization. The crux of the new proof is an upper bound constructed via the convex conjugate of the objective function.