A Scalable Frank-Wolfe based Augmented Lagrangian Method for Linearly Constrained Composite Convex Programming

TL;DR

This paper introduces a scalable Frank-Wolfe based Augmented Lagrangian method tailored for large-scale linearly constrained composite convex optimization, offering efficient per-iteration costs and proven convergence rates.

Contribution

It presents a novel FW-AL method that efficiently handles large-scale problems by combining Frank-Wolfe and augmented Lagrangian techniques with convergence analysis.

Findings

Method scales linearly with problem size

Achieves non-ergodic convergence rate

Suitable for large-scale convex optimization

Abstract

In this paper, we consider large-scale linearly constrained composite convex optimization problem, whose objective is a sum of a smooth function and a possibly nonsmooth function. We propose a scalable \textbf{F}rank-\textbf{W}olfe based \textbf{A}ugmented \textbf{L}agrangian (FW-AL) method for solving this problem. At each iteration, the proposed FW-AL method employs the FW method (or its variants) to approximately solve the AL subproblem {(with fixed Lagrange multiplier)} within a preselected tolerance and then updates the Lagrange multiplier. The proposed FW-AL method is well suitable for solving large-scale problems, because its computational cost per step scales (essentially) linearly with the size of the input. We analyze the non-ergodic convergence rate of the proposed FW-AL method.