Iteration Complexity of Frank-Wolfe and Its Variants for Bilevel Optimization

TL;DR

This paper analyzes the iteration complexity of Frank-Wolfe algorithms and their variants for constrained bilevel optimization, especially when the lower-level problem is solved approximately, providing convergence guarantees and practical validation.

Contribution

It introduces convergence analysis for inexact Frank-Wolfe variants in bilevel optimization, combining theoretical bounds with real-world experiments.

Findings

Convergence rates established for inexact Frank-Wolfe methods.

Iteration complexity guarantees for bilevel problems.

Experimental validation on real-world applications.

Abstract

We study Frank-Wolfe (FW) methods for constrained bilevel optimization when the lower-level problem is solved only approximately, yielding biased and inexact hypergradients. We analyze inexact variants of vanilla FW as well as away-step and pairwise FW, and provide convergence rates in the nonconvex setting under gradient errors. By combining these results with recent bounds on hypergradient errors from iterative and approximate implicit differentiation, we derive overall iteration complexity guarantees for bilevel FW. Experiments on two real-world applications validate the theory and demonstrate practical effectiveness.