Robustness of the Frank-Wolfe Method under Inexact Oracles and the Cost of Linear Minimization

TL;DR

This paper analyzes the robustness of the Frank-Wolfe optimization method when using inexact gradients and compares the computational costs of linear minimization oracles versus projections, providing convergence guarantees and efficiency insights.

Contribution

It extends convergence analysis of Frank-Wolfe to inexact oracles for nonconvex functions and shows that approximate projections are not cheaper than LMOs, highlighting robustness and efficiency.

Findings

Convergence guarantee of order O(1/√k + δ) for inexact Frank-Wolfe on nonconvex functions.

Oracle errors do not accumulate asymptotically, ensuring robustness.

Approximate projections are not computationally cheaper than accurate LMOs.

Abstract

We investigate the robustness of the Frank-Wolfe method when gradients are computed inexactly and examine the relative computational cost of the linear minimization oracle (LMO) versus projection. For smooth nonconvex functions, we establish a convergence guarantee of order $\mathcal{O}(1/\sqrt{k}+\delta)$ for Frank-Wolfe with a $\delta$--oracle. Our results strengthen previous analyses for convex objectives and show that the oracle errors do not accumulate asymptotically. We further prove that approximate projections cannot be computationally cheaper than accurate LMOs, thus extending to the case of inexact projections. These findings reinforce the robustness and efficiency of the Frank-Wolfe framework.