Active set complexity of the Away-step Frank-Wolfe Algorithm

TL;DR

This paper analyzes the active set identification capabilities of the away-step Frank-Wolfe algorithm, providing convergence rates and complexity bounds for both convex and nonconvex problems, with extensions to general polytopes.

Contribution

It introduces new active set identification results and convergence rates for the away-step Frank-Wolfe algorithm in nonconvex settings, including explicit complexity bounds.

Findings

Active set identification achieved under certain conditions.

Novel $O(1/\sqrt{k})$ convergence rate for nonconvex problems.

Explicit active set complexity bounds for convex and nonconvex objectives.

Abstract

In this paper, we study active set identification results for the away-step Frank-Wolfe algorithm in different settings. We first prove a local identification property that we apply, in combination with a convergence hypothesis, to get an active set identification result. We then prove, in the nonconvex case, a novel $O(1/\sqrt{k})$ convergence rate result and active set identification for different stepsizes (under suitable assumptions on the set of stationary points). By exploiting those results, we also give explicit active set complexity bounds for both strongly convex and nonconvex objectives. While we initially consider the probability simplex as feasible set, in the appendix we show how to adapt some of our results to generic polytopes.