Frank-Wolfe and friends: a journey into projection-free first-order optimization methods

TL;DR

This paper reviews the Frank-Wolfe optimization method, highlighting its recent resurgence due to its simplicity, versatility, and improvements in speed and efficiency in data science applications.

Contribution

It provides a comprehensive overview of the Frank-Wolfe method's history, recent variants, and its broad applicability across various fields.

Findings

Frank-Wolfe method is versatile and applicable in many contexts.

Recent variants improve speed and efficiency.

The method's simplicity contributes to its success.

Abstract

Invented some 65 years ago in a seminal paper by Marguerite Straus-Frank and Philip Wolfe, the Frank-Wolfe method recently enjoys a remarkable revival, fuelled by the need of fast and reliable first-order optimization methods in Data Science and other relevant application areas. This review tries to explain the success of this approach by illustrating versatility and applicability in a wide range of contexts, combined with an account on recent progress in variants, both improving on the speed and efficiency of this surprisingly simple principle of first-order optimization.