Complexity of Linear Minimization and Projection on Some Sets

TL;DR

This paper reviews the computational complexity of linear minimization and projection tasks on various sets, highlighting the advantages and providing new projection methods for specific sets used in optimization.

Contribution

It offers a comprehensive analysis of complexity bounds and introduces projection algorithms for the all and Birkhoff polytope, clarifying the efficiency of the Frank-Wolfe algorithm.

Findings

Complexity bounds for linear minimization and projection on common sets

New projection algorithms for all and Birkhoff polytope

Clarification of the computational advantages of linear minimizations

Abstract

The Frank-Wolfe algorithm is a method for constrained optimization that relies on linear minimizations, as opposed to projections. Therefore, a motivation put forward in a large body of work on the Frank-Wolfe algorithm is the computational advantage of solving linear minimizations instead of projections. However, the discussions supporting this advantage are often too succinct or incomplete. In this paper, we review the complexity bounds for both tasks on several sets commonly used in optimization. Projection methods onto the $\ell_p$-ball, $p\in\left]1,2\right[\cup\left]2,+\infty\right[$, and the Birkhoff polytope are also proposed.