Auto-conditioned primal-dual hybrid gradient method and alternating direction method of multipliers

TL;DR

This paper introduces the auto-conditioned primal-dual hybrid gradient (AC-PDHG) and ADMM (AC-ADMM) methods, which adaptively solve bilinear saddle point and linearly constrained problems without line search, achieving optimal complexity and convergence guarantees.

Contribution

The paper presents novel adaptive primal-dual and ADMM algorithms that eliminate the need for line search, with proven optimal complexity and convergence guarantees for bilinear and linearly constrained problems.

Findings

AC-PDHG achieves optimal complexity without line search.

AC-ADMM guarantees convergence based on partial constraint information.

Both methods extend to problems with additional smooth terms and incorporate acceleration.

Abstract

Line search procedures are often employed in primal-dual methods for bilinear saddle point problems, especially when the norm of the linear operator is large or difficult to compute. In this paper, we demonstrate that line search is unnecessary by introducing a novel primal-dual method, the auto-conditioned primal-dual hybrid gradient (AC-PDHG) method, which achieves optimal complexity for solving bilinear saddle point problems. AC-PDHG is fully adaptive to the linear operator, using only past iterates to estimate its norm. We further tailor AC-PDHG to solve linearly constrained problems, providing convergence guarantees for both the optimality gap and constraint violation. Moreover, we explore an important class of linearly constrained problems where both the objective and constraints decompose into two parts. By incorporating the design principles of AC-PDHG into the preconditioned alternating direction method of multipliers (ADMM), we propose the auto-conditioned alternating direction method of multipliers (AC-ADMM), which guarantees convergence based solely on one part of the constraint matrix and fully adapts to it, eliminating the need for line search. Finally, we extend both AC-PDHG and AC-ADMM to solve bilinear problems with an additional smooth term. By integrating these methods with a novel acceleration scheme, we attain optimal iteration complexities under the single-oracle setting.