A Generalized Frank-Wolfe Method With "Dual Averaging" for Strongly Convex Composite Optimization

TL;DR

This paper introduces a simple Frank-Wolfe variant with dual averaging that achieves linear convergence for strongly convex problems and provides new convergence guarantees for the logistic fictitious play algorithm.

Contribution

It proposes a novel Frank-Wolfe variant with dual averaging, establishing linear convergence and analyzing local convergence rates of logistic fictitious play.

Findings

Achieves linear convergence rate on duality gaps.

Provides probabilistic local convergence rate for logistic fictitious play.

Demonstrates effectiveness of the method on strongly convex composite problems.

Abstract

We propose a simple variant of the generalized Frank-Wolfe method for solving strongly convex composite optimization problems, by introducing an additional averaging step on the dual variables. We show that in this variant, one can choose a simple constant step-size and obtain a linear convergence rate on the duality gaps. By leveraging the convergence analysis of this variant, we then analyze the local convergence rate of the logistic fictitious play algorithm, which is well-established in game theory but lacks any form of convergence rate guarantees. We show that, with high probability, this algorithm converges locally at rate $O(1/t)$, in terms of certain expected duality gap.