A first-order primal-dual algorithm with linesearch

TL;DR

This paper introduces a linesearch technique for primal-dual algorithms that improves convergence rates and reduces computational costs, especially for regularized least squares problems, with proven theoretical guarantees and numerical validation.

Contribution

It presents a novel linesearch method for primal-dual algorithms that requires minimal additional computation and achieves improved convergence rates under certain conditions.

Findings

Proves convergence of the proposed linesearch method.

Achieves an ergodic $O(1/N)$ convergence rate.

Numerical experiments demonstrate efficiency improvements.

Abstract

The paper proposes a linesearch for a primal-dual method. Each iteration of the linesearch requires to update only the dual (or primal) variable. For many problems, in particular for regularized least squares, the linesearch does not require any additional matrix-vector multiplications. We prove convergence of the proposed method under standard assumptions. We also show an ergodic $O(1/N)$ rate of convergence for our method. In case one or both of the prox-functions are strongly convex, we modify our basic method to get a better convergence rate. Finally, we propose a linesearch for a saddle point problem with an additional smooth term. Several numerical experiments confirm the efficiency of our proposed methods.