Convergence analysis of primal-dual augmented Lagrangian methods and duality theory

TL;DR

This paper presents a unified theoretical framework for augmented Lagrangian methods in nonconvex optimization, linking duality concepts with convergence analysis in infinite-dimensional spaces.

Contribution

It unifies duality theory and convergence analysis for augmented Lagrangians, highlighting the role of duality concepts in convergence properties and applicability to various problem types.

Findings

Zero duality gap is necessary for primal sequence boundedness.

Existence of an optimal dual solution is necessary for bounded multipliers.

Results apply to multiple augmented Lagrangian formulations.

Abstract

We develop a unified theory of augmented Lagrangians for nonconvex optimization problems that encompasses both duality theory and convergence analysis of primal-dual augmented Lagrangian methods in the infinite dimensional setting. Our goal is to present many well-known concepts and results related to augmented Lagrangians in a unified manner and bridge a gap between existing convergence analysis of primal-dual augmented Lagrangian methods and abstract duality theory. Within our theory we specifically emphasize the role of various fundamental duality concepts (such as duality gap, optimal dual solutions, global saddle points, etc.) in convergence analysis of augmented Lagrangians methods and underline interconnections between all these concepts and convergence of primal and dual sequences generated by such methods. In particular, we prove that the zero duality gap property is a necessary condition for the boundedness of the primal sequence, while the existence of an optimal dual solution is a necessary condition for the boundedness of the sequences of multipliers and penalty parameters, irrespective of the way in which the multipliers and the penalty parameter are updated. Our theoretical results are applicable to many different augmented Lagrangians for various types of cone constrained optimization problems, including Rockafellar-Wets' augmented Lagrangian, (penalized) exponential/hyperbolic-type augmented Lagrangians, modified barrier functions, etc.