A conditional-gradient-based single-loop augmented Lagrangian method for inequality constrained optimization

TL;DR

This paper introduces a novel single-loop augmented Lagrangian method using conditional gradient steps for inequality constrained convex optimization, achieving optimal convergence rates.

Contribution

It adapts the classical augmented Lagrangian method with a conditional gradient approach, providing convergence guarantees and accelerated rates under certain conditions.

Findings

Convergence rate matches the best-known complexity for the problem class.

Achieves accelerated rates when $h$ is an indicator of a uniformly convex set.

Validates the effectiveness of the method under standard stepsize rules.

Abstract

We consider the problem of minimizing the sum of a Lipschitz differentiable convex function $f$ and a proper closed convex function $h$ that admits efficient linear minimization oracles, subject to multiple smooth convex inequality constraints. We adapt the classical augmented Lagrangian (AL) method for these problems: in each iteration, our algorithm consists of one step of the conditional gradient (CG) method applied to the AL function, followed by an update of the dual variable as in classical AL methods with a diminishing dual stepsize. We study the convergence rate of our algorithm under two standard stepsize rules for the CG method, namely, an open-loop stepsize and the short stepsize, and obtain a convergence rate that matches the best-known complexity for this class of problems. We also establish accelerated rates when $h$ is the indicator function of a uniformly convex set.