Frank-Wolfe Algorithms for Saddle Point Problems

TL;DR

This paper extends the Frank-Wolfe algorithm to efficiently solve constrained convex-concave saddle point problems, providing the first convergence proof over polytopes and exploring applications in structured prediction and game theory.

Contribution

It introduces a Frank-Wolfe based saddle point solver with proven convergence over polytopes, addressing a long-standing open problem in optimization.

Findings

First convergence proof of FW-type saddle point solver over polytopes.

Survey of existing convergence results and identification of theoretical gaps.

Application to structured prediction and combinatorial game problems.

Abstract

We extend the Frank-Wolfe (FW) optimization algorithm to solve constrained smooth convex-concave saddle point (SP) problems. Remarkably, the method only requires access to linear minimization oracles. Leveraging recent advances in FW optimization, we provide the first proof of convergence of a FW-type saddle point solver over polytopes, thereby partially answering a 30 year-old conjecture. We also survey other convergence results and highlight gaps in the theoretical underpinnings of FW-style algorithms. Motivating applications without known efficient alternatives are explored through structured prediction with combinatorial penalties as well as games over matching polytopes involving an exponential number of constraints.