Lower Bounds for Frank-Wolfe on Strongly Convex Sets

TL;DR

This paper establishes a tight lower bound of er or the convergence rate of the Frank-Wolfe algorithm on strongly convex sets, showing it cannot generally outperform er or smooth, strongly convex problems.

Contribution

The paper provides the first constructive lower bound for Frank-Wolfe on strongly convex sets, demonstrating the er or rate is tight and exploring conditions for faster convergence.

Findings

Frank-Wolfe has a er or lower bound of er or on strongly convex sets.

Develops a novel computational approach to construct worst-case trajectories.

Proves that strong convexity of the set alone does not guarantee faster convergence rates.

Abstract

We present a constructive lower bound of $\Omega(1/\sqrt{\varepsilon})$ for Frank-Wolfe (FW) when both the objective and the constraint set are smooth and strongly convex, showing that the known uniform $\mathcal{O}(1/\sqrt{\varepsilon})$ guarantees in this regime are tight. It is known that under additional assumptions on the position of the optimizer, FW can converge linearly. However, it remained unclear whether strong convexity of the set can yield rates uniformly faster than $\mathcal{O}(1/\sqrt{\varepsilon})$, i.e., irrespective of the position of the optimizer. To investigate this question, we focus on a simple yet representative problem class: minimizing a strongly convex quadratic over the Euclidean unit ball, with the optimizer on the boundary. We analyze the dynamics of FW for this problem in detail and develop a novel computational approach to construct worst-case FW trajectories, which is of independent interest. Guided by these constructions, we develop an analytical proof establishing the lower bound.