Auto-Conditioned Frank-Wolfe Algorithms

TL;DR

This paper introduces an auto-conditioned Frank-Wolfe framework that adapts step sizes based on local curvature, improving practical performance without needing prior smoothness knowledge.

Contribution

It develops a unified, local Lipschitz-based step-size scheme applicable to various Frank-Wolfe variants, achieving convergence guarantees and practical improvements.

Findings

Convergence to stationary points in nonconvex settings.

Standard sublinear convergence in convex cases.

Numerical experiments show significant practical gains.

Abstract

Frank-Wolfe methods are projection-free algorithms for constrained optimization whose practical performance often depends critically on the choice of step size. Classical closed-loop step-size rules typically require prior knowledge of a global smoothness constant, while line-search variants avoid this requirement at the cost of additional function evaluations and implementation overhead. In this paper, we develop a fully auto-conditioned framework for Frank-Wolfe-type methods. The framework replaces the global Lipschitz constant in closed-loop step sizes with a local Lipschitz estimator computed from first-order information along the iterates. We show that this abstraction captures several important projection-free subroutines, including standard Frank-Wolfe, Matching Pursuit, pairwise Frank-Wolfe, and away-step Frank-Wolfe. For the resulting general class of methods, we establish convergence to stationary points in the nonconvex setting and recover the standard sublinear convergence guarantees in the convex setting, without requiring prior knowledge of a global smoothness constant. We further show that, when specialized to particular Frank-Wolfe variants and combined with additional structural assumptions, the same auto-conditioned framework yields accelerated convergence rates. Numerical experiments demonstrate that the proposed methods provide substantial practical improvements over line-search-based alternatives, highlighting the benefits of adapting to local curvature while retaining the simplicity of closed-loop step-size rules.