Quadratic error bound of the smoothed gap and the restarted averaged primal-dual hybrid gradient

TL;DR

This paper introduces the quadratic error bound of the smoothed gap as a new regularity condition, enabling tighter convergence analysis of the primal-dual hybrid gradient method, with practical benefits demonstrated through numerical experiments.

Contribution

The paper proposes the quadratic error bound of the smoothed gap, a novel regularity assumption, and shows how averaging and restarting improve convergence rates under this condition.

Findings

Tighter convergence rates for the primal-dual hybrid gradient method.

Numerical validation on linear, quadratic, and imaging problems.

Enhanced understanding of the method's behavior with the new regularity assumption.

Abstract

We study the linear convergence of the primal-dual hybrid gradient method. After a review of current analyses, we show that they do not explain properly the behavior of the algorithm, even on the most simple problems. We thus introduce the quadratic error bound of the smoothed gap, a new regularity assumption that holds for a wide class of optimization problems. Equipped with this tool, we manage to prove tighter convergence rates. Then, we show that averaging and restarting the primal-dual hybrid gradient allows us to leverage better the regularity constant. Numerical experiments on linear and quadratic programs, ridge regression and image denoising illustrate the findings of the paper.