Optimal Acceleration for Proximal Minimization of the Sum of Convex and Strongly Convex Functions

TL;DR

This paper introduces Fast Douglas--Rachford Splitting (FDR), an accelerated method for convex optimization that achieves an optimal convergence rate and improves constants over previous methods.

Contribution

The paper presents FDR, an accelerated algorithm with proven optimal convergence rate and constants for minimizing sums of convex and strongly convex functions.

Findings

FDR achieves the $oxed{ ext{O}(1/N^2)}$ convergence rate.

The convergence rate and leading constants of FDR are proven to be optimal.

FDR improves upon previous accelerated methods in constants.

Abstract

When minimizing the sum of a convex and a strongly convex function, or when finding the zero of the sum of a monotone operator and a strongly monotone operator, Chambolle and Pock (2010) and Davis and Yin (2015) proposed accelerated mechanisms that achieve an $\mathcal{O}(1/N^2)$ convergence rate for the squared distance to the solution, but the optimality of this rate was not established. In this work, we present Fast Douglas--Rachford Splitting (FDR), an accelerated method that improves the constants established in the prior works, and provide a complexity lower bound establishing that both the $\mathcal{O}(1/N^2)$ convergence rate and the leading-order constant of FDR's rate are optimal.