Projection-free Online Learning with Arbitrary Delays

TL;DR

This paper extends projection-free online learning algorithms to handle arbitrary delays in gradient information, maintaining optimal regret bounds even with significant delays, thus broadening their practical applicability.

Contribution

It generalizes the online Frank-Wolfe and OSPF algorithms to settings with arbitrary gradient delays, preserving their regret guarantees.

Findings

Algorithms maintain optimal regret bounds despite delays.

Delayed gradient handling is effectively integrated into existing algorithms.

Theoretical analysis confirms robustness to large delays.

Abstract

Projection-free online learning, which eschews the projection operation via less expensive computations such as linear optimization (LO), has received much interest recently due to its efficiency in handling high-dimensional problems with complex constraints. However, previous studies assume that any queried gradient is revealed immediately, which may not hold in practice and limits their applications. To address this limitation, we generalize the online Frank-Wolfe (OFW) algorithm and the online smooth projection-free (OSPF) algorithm, which are state-of-the-art LO-based projection-free online algorithms for non-smooth and smooth functions respectively, into a delayed setting where queried gradients can be delayed by arbitrary rounds. Specifically, the main idea of our generalized OFW is to perform an update similar to the original OFW after receiving any delayed gradient, and play the latest decision for each round. Moreover, the essential change on OSPF is to replace the sum of queried gradients, which is originally utilized in each update, with the sum of available gradients. Despite their simplicities, our novel analysis shows that under a relatively large amount of delay, the generalized OFW and OSPF enjoy the same regret bound as OFW and OSPF in the non-delayed setting, respectively.