Secant Line Search for Frank-Wolfe Algorithms

TL;DR

This paper introduces a secant line search strategy for Frank-Wolfe algorithms that adapts to local smoothness, offering similar effectiveness to full line search but with lower computational cost and improved convergence rates.

Contribution

A novel secant-based step-size method for Frank-Wolfe algorithms that reduces computational cost while maintaining convergence efficiency.

Findings

Comparable effectiveness to full line search in experiments

Reduced computational cost compared to traditional line search

Theoretical guarantees established for the new strategy

Abstract

We present a new step-size strategy based on the secant method for Frank-Wolfe algorithms. This strategy, which requires mild assumptions about the function under consideration, can be applied to any Frank-Wolfe algorithm. It is as effective as full line search and, in particular, allows for adapting to the local smoothness of the function, such as in Pedregosa et al 2018, but comes with a significantly reduced computational cost, leading to higher effective rates of convergence. We provide theoretical guarantees and demonstrate the effectiveness of the strategy through numerical experiments.