Primal-Dual Bundle Methods for Linear Equality-Constrained Problems

TL;DR

This paper introduces primal-dual bundle methods that enhance convergence speed and robustness for linear equality-constrained convex optimization by using more accurate surrogate functions in dual ascent and method of multipliers.

Contribution

The paper develops a new family of primal-dual bundle methods that generalize existing algorithms and improve convergence and robustness through advanced surrogate functions.

Findings

Faster convergence compared to existing methods

Enhanced robustness to parameter choices

Theoretical convergence guarantees provided

Abstract

Dual ascent (DA) and the method of multipliers (MM) are fundamental methods for solving linear equality-constrained convex optimization problems, and their dual updates can be viewed as the minimization of a proximal linear surrogate function of the negative Lagrange dual and augmented Lagrange dual function, respectively. However, the proximal linear surrogate function may suffer from low approximation accuracy, which leads to slow convergence of DA and MM. To accelerate their convergence, we adapt the proximal bundle surrogate framework that can incorporate a list of more accurate surrogate functions, to both the primal and the dual updates of DA and MM, leading to a family of novel primal-dual bundle methods. Our methods generalize the primal-dual gradient method, DA, the linearized MM, and MM. Under standard assumptions that allow for a broad range of surrogate functions, we prove theoretical convergence guarantees for the proposed methods. Numerical experiments demonstrate that our methods converge not only faster, but also significantly more robust with respect to the parameters compared to the primal-dual gradient method, DA, the linearized MM, and MM.