Smoothness of the Augmented Lagrangian Dual in Convex Optimization

TL;DR

This paper proves that the augmented Lagrangian dual function is smooth everywhere under mild conditions, broadening the theoretical understanding of convex optimization duality.

Contribution

It establishes the universal smoothness of the augmented Lagrangian dual function under minimal assumptions, relaxing previous stringent conditions.

Findings

The augmented Lagrangian dual function is $rac{1}{ ho}$-smooth everywhere.

Existence of solutions to the augmented Lagrangian minimization for any dual variable.

Weakening of traditional assumptions needed for dual function smoothness.

Abstract

This paper focuses on the general linearly constrained optimization problem: $\min_{x \in \mathbb{R}^d} f(x) \ \text{s.t.} \ Ax = b$, where $f: \mathbb{R}^d \rightarrow \mathbb{R} \cup \{+\infty\}$ is a closed proper convex function, $A \in \mathbb{R}^{p \times d}$, and $b \in \mathbb{R}^p$. We define the standard dual function $\phi(\lambda) = \inf_x \{f(x) + \langle \lambda, A x - b \rangle\}$, the augmented Lagrangian $\mathcal{L}_{\rho}(x, \lambda) = f(x) + \langle \lambda, Ax - b \rangle + \frac{\rho}{2}\|Ax - b\|^2$ ($\rho > 0$), and the augmented Lagrangian dual function $\phi_{\rho}(\lambda) = \inf_x \mathcal{L}_{\rho}(x, \lambda)$. Under the fundamental condition that $\text{dom} \ \phi \neq \emptyset$, we establish that: (1) $\phi_{\rho}$ is $\frac{1}{\rho}$-smooth everywhere; and (2) the solution to $\min_{x \in \mathbb{R}^d} \mathcal{L}_{\rho}(x, \lambda)$ exists for any $\lambda \in \mathbb{R}^p$. These theoretical findings substantially weaken the stringent assumptions typically imposed in the literature to ensure such properties.