Frank-Wolfe Style Algorithms for Large Scale Optimization

TL;DR

This paper presents modified Frank-Wolfe algorithms incorporating stochastic gradients, approximations, and sketching techniques to efficiently handle large-scale optimization problems while maintaining optimal convergence rates.

Contribution

It introduces new variants of Frank-Wolfe algorithms tailored for large-scale problems with scalable modifications and preserved convergence guarantees.

Findings

Achieved scalable Frank-Wolfe algorithms with stochastic and approximate methods.

Maintained the optimal convergence rate of /k in large-scale settings.

Demonstrated effectiveness on enormous optimization problems.

Abstract

We introduce a few variants on Frank-Wolfe style algorithms suitable for large scale optimization. We show how to modify the standard Frank-Wolfe algorithm using stochastic gradients, approximate subproblem solutions, and sketched decision variables in order to scale to enormous problems while preserving (up to constants) the optimal convergence rate $\mathcal{O}(\frac{1}{k})$.