Efficient Quadratic Corrections for Frank-Wolfe Algorithms

TL;DR

This paper introduces efficient quadratic correction methods for Frank-Wolfe algorithms, improving convergence and computational speed for convex optimization problems, especially those with quadratic objectives.

Contribution

It generalizes existing Frank-Wolfe variants with corrective steps, proposes two new quadratic correction techniques, and demonstrates their effectiveness through theoretical guarantees and empirical speedups.

Findings

Proves tight convergence guarantees for the new algorithms.

Develops two efficient quadratic correction steps based on linear optimization and linear system solving.

Shows substantial computational speedups in various problem classes.

Abstract

We develop a Frank-Wolfe algorithm with corrective steps, generalizing previous algorithms including blended conditional gradients, blended pairwise conditional gradients, and fully-corrective Frank-Wolfe. For this, we prove tight convergence guarantees together with an optimal face identification property. Furthermore, we propose two highly efficient corrective steps for convex quadratic objectives based on linear optimization or linear system solving, akin to Wolfe's minimum-norm point, and show that they converge in finite time under suitable conditions. Beyond optimization problems that are directly quadratic, we revisit two algorithms - split conditional gradient and second-order conditional gradient sliding - which can leverage quadratic corrections to accelerate their quadratic subproblems. We demonstrate improved convergence rates for the first and broader applicability for the second, which may be of independent interest. Finally, we show substantial computational speedups for Frank-Wolfe-based algorithms with quadratic corrections across the considered problem classes.