Fast convergence of Frank-Wolfe algorithms on polytopes

TL;DR

This paper introduces a unified framework to derive convergence rates for various Frank-Wolfe algorithms on polytopes, based on affine-invariant properties like error bounds and curvature.

Contribution

It provides a template linking convergence rates to affine-invariant properties, applicable to multiple Frank-Wolfe variants on polytopes, regardless of norms.

Findings

Derived convergence rates from sublinear to linear for different algorithms.

Rates depend solely on polytope and objective function properties.

Unified analysis simplifies understanding of Frank-Wolfe algorithm performance.

Abstract

We provide a template to derive convergence rates for the following popular versions of the Frank-Wolfe algorithm on polytopes: vanilla Frank-Wolfe, Frank-Wolfe with away steps, Frank-Wolfe with blended pairwise steps, and Frank-Wolfe with in-face directions. Our template shows how the convergence rates follow from two affine-invariant properties of the problem, namely, error bound and extended curvature. These properties depend solely on the polytope and objective function but not on any affine-dependent object like norms. For each one of the above algorithms, we derive rates of convergence ranging from sublinear to linear depending on the degree of the error bound.