Frank-Wolfe Beyond 1/t Convergence

TL;DR

This paper introduces a new condition called Local Dual Sharpness (LDS) that guarantees the Frank-Wolfe algorithm converges faster than the traditional 1/t rate for smooth convex minimization over certain sets.

Contribution

The paper establishes the LDS condition under which Frank-Wolfe achieves convergence rates faster than 1/t, extending understanding of convergence behavior beyond classical bounds.

Findings

LDS condition ensures o(1/t) convergence for Frank-Wolfe.

Uniformly convex sets satisfy LDS, leading to unconditional faster convergence.

Combining LDS with local H"older error bounds quantifies improved rates.

Abstract

We consider smooth convex minimization over compact convex sets, i.e., $\min_{x \in C} f(x)$ with the (vanilla) Frank-Wolfe algorithm. Well-known lower bounds establish a worst-case $\Omega(1/t)$ primal-gap barrier in the general smooth convex case, and faster convergence usually requires favorable function properties such as H\"older error bounds or strong convexity. We present a new Local Dual Sharpness (LDS) condition, essentially a property of the feasible region and its LMO, under which the Frank-Wolfe algorithm converges in $o(1/t)$ for any smooth convex function, ruling out an $\Omega(1/t)$ lower bound under LDS. The condition is a generalization (and localization) of uniform convexity of sets and it is satisfied by any uniformly convex set. To our knowledge, this is the first unconditional $o(1/t)$ convergence result for uniformly convex sets. Combining LDS with stronger function properties, e.g., a local variant of H\"older error bounds, allows us to quantify the actual rates.