Faster Rates for the Frank-Wolfe Algorithm Using Jacobi Polynomials

TL;DR

This paper introduces a novel acceleration technique for the Frank-Wolfe algorithm using Jacobi polynomials, achieving faster convergence rates for large-scale convex optimization problems.

Contribution

It extends the Frank-Wolfe algorithm by incorporating Jacobi polynomial-based acceleration, providing a new method for faster convergence.

Findings

Faster sublinear convergence rate achieved.

Numerical experiments confirm improved performance.

Jacobi polynomial parameters are crucial for acceleration.

Abstract

The Frank Wolfe algorithm (FW) is a popular projection-free alternative for solving large-scale constrained optimization problems. However, the FW algorithm suffers from a sublinear convergence rate when minimizing a smooth convex function over a compact convex set. Thus, exploring techniques that yield a faster convergence rate becomes crucial. A classic approach to obtain faster rates is to combine previous iterates to obtain the next iterate. In this work, we extend this approach to the FW setting and show that the optimal way to combine the past iterates is using a set of orthogonal Jacobi polynomials. We also a polynomial-based acceleration technique, referred to as Jacobi polynomial accelerated FW, which combines the current iterate with the past iterate using combing weights related to the Jacobi recursion. By carefully choosing parameters of the Jacobi polynomials, we obtain a faster sublinear convergence rate. We provide numerical experiments on real datasets to demonstrate the efficacy of the proposed algorithm.