Indefinite linearized augmented Lagrangian method for convex programming with linear inequality constraints

TL;DR

This paper introduces a new indefinite linearized augmented Lagrangian method tailored for convex programming problems with linear inequality constraints, offering improved efficiency and convergence properties especially for large-scale problems.

Contribution

It develops a novel indefinite linearized ALM scheme that simplifies subproblems and allows larger step-sizes, with proven global convergence and an $ ext{O}(1/N)$ rate.

Findings

Method achieves better performance with smaller regularization terms.

Global convergence with $ ext{O}(1/N)$ rate is established.

Numerical results confirm theoretical advantages.

Abstract

The augmented Lagrangian method (ALM) is a benchmark for convex programming problems with linear constraints; ALM and its variants for linearly equality-constrained convex minimization models have been well studied in the literature. However, much less attention has been paid to ALM for efficiently solving linearly inequality-constrained convex minimization models. In this paper, we exploit an enlightening reformulation of the newly developed indefinite linearized ALM for the equality-constrained convex optimization problem, and present a new indefinite linearized ALM scheme for efficiently solving the convex optimization problem with linear inequality constraints. The proposed method enjoys great advantages, especially for large-scale optimization cases, in two folds mainly: first, it largely simplifies the challenging key subproblem of the classic ALM by employing its linearized reformulation, while keeping low complexity in computation; second, we show that only a smaller proximity regularization term is needed for provable convergence, which allows a bigger step-size and hence significantly better performance. Moreover, we show the global convergence of the proposed scheme upon its equivalent compact expression of prediction-correction, along with a worst-case $\mathcal{O}(1/N)$ convergence rate. Numerical results on some application problems demonstrate that a smaller regularization term can lead to a better experimental performance, which further confirms the theoretical results presented in this study.