Inexact First-Order Primal-Dual Algorithms

TL;DR

This paper analyzes the convergence of inexact primal-dual algorithms for saddle point problems, demonstrating that they retain optimal convergence rates under certain error decay conditions and applying them to nested algorithms.

Contribution

It provides convergence guarantees for inexact primal-dual algorithms with errors, extending the understanding of their robustness and practical applicability.

Findings

Optimal $O(1/N)$ convergence in finite dimensions for non-smooth problems

Linear $O( heta^N)$ convergence with strong convexity

Slower convergence rates under less strict error decay conditions

Abstract

In this paper we investigate the convergence of a recently popular class of first-order primal-dual algorithms for saddle point problems under the presence of errors occurring in the proximal maps and gradients. We study several types of errors and show that, provided a sufficient decay of these errors, the same convergence rates as for the error-free algorithm can be established. More precisely, we prove the (optimal) $O(1/N)$ convergence to a saddle point in finite dimensions for the class of non-smooth problems considered in this paper, and prove a $O(1/N^2)$ or even linear $O(\theta^N)$ convergence rate if either the primal or dual objective respectively both are strongly convex. Moreover we show that also under a slower decay of errors we can establish rates, however slower and directly depending on the decay of the errors. We demonstrate the performance and practical use of the algorithms on the example of nested algorithms and show how they can be used to split the global objective more efficiently.