The Pivoting Framework: Frank-Wolfe Algorithms with Active Set Size Control

TL;DR

This paper introduces the pivoting meta algorithm (PM) that controls the active set size in Frank-Wolfe optimization methods, maintaining convergence rates while reducing computational complexity.

Contribution

The paper presents PM, a novel meta algorithm that enforces active set size control in Frank-Wolfe variants through an efficient linear program reformulation.

Findings

PM maintains active set size at most dimension plus one.

PM preserves the convergence rate of original algorithms.

Numerical experiments demonstrate practical active set size reduction.

Abstract

We propose the pivoting meta algorithm (PM) to enhance optimization algorithms that generate iterates as convex combinations of vertices of a feasible region $C\subseteq \mathbb{R}^n$, including Frank-Wolfe (FW) variants. PM guarantees that the active set (the set of vertices in the convex combination) of the modified algorithm remains as small as $\mathrm{dim}(C)+1$ as stipulated by Carath\'eodory's theorem. PM achieves this by reformulating the active set expansion task into an equivalent linear program, which can be efficiently solved using a single pivot step akin to the primal simplex algorithm; the convergence rate of the original algorithms are maintained. Furthermore, we establish the connection between PM and active set identification, in particular showing under mild assumptions that PM applied to the away-step Frank-Wolfe algorithm or the blended pairwise Frank-Wolfe algorithm bounds the active set size by the dimension of the optimal face plus $1$. We provide numerical experiments to illustrate practicality and efficacy on active set size reduction.