Convergence Rate of Frank-Wolfe for Non-Convex Objectives

TL;DR

This paper proves that the Frank-Wolfe algorithm converges to a stationary point at a rate of O(1/√t) for non-convex functions with Lipschitz continuous gradients, matching known rates for projected gradient methods.

Contribution

It provides a simple, affine-invariant proof establishing the convergence rate of Frank-Wolfe for non-convex objectives, a novel result in the field.

Findings

Frank-Wolfe achieves O(1/√t) convergence rate on non-convex problems.

The proof is affine invariant and straightforward.

First known rate similar to projected gradient methods for this setting.

Abstract

We give a simple proof that the Frank-Wolfe algorithm obtains a stationary point at a rate of $O(1/\sqrt{t})$ on non-convex objectives with a Lipschitz continuous gradient. Our analysis is affine invariant and is the first, to the best of our knowledge, giving a similar rate to what was already proven for projected gradient methods (though on slightly different measures of stationarity).