Online Prediction in Sub-linear Space

TL;DR

This paper introduces the first sub-linear space and regret algorithm for online expert advice, demonstrating a separation between oblivious and adaptive adversaries, and providing new lower bounds and reduction techniques.

Contribution

It presents a novel sub-linear space, sub-linear regret algorithm for online learning with expert advice and establishes a lower bound against adaptive adversaries.

Findings

First sub-linear space and regret algorithm for online learning with expert advice.

Proves a linear memory lower bound for adaptive adversaries.

Introduces a reduction from weakly sub-linear to polynomial regret algorithms.

Abstract

We provide the first sub-linear space and sub-linear regret algorithm for online learning with expert advice (against an oblivious adversary), addressing an open question raised recently by Srinivas, Woodruff, Xu and Zhou (STOC 2022). We also demonstrate a separation between oblivious and (strong) adaptive adversaries by proving a linear memory lower bound of any sub-linear regret algorithm against an adaptive adversary. Our algorithm is based on a novel pool selection procedure that bypasses the traditional wisdom of leader selection for online learning, and a generic reduction that transforms any weakly sub-linear regret $o(T)$ algorithm to $T^{1-\alpha}$ regret algorithm, which may be of independent interest. Our lower bound utilizes the connection of no-regret learning and equilibrium computation in zero-sum games, leading to a proof of a strong lower bound against an adaptive adversary.