Replicable Online Learning

TL;DR

This paper introduces the concept of adversarially replicable online learning algorithms that produce identical actions across independent runs, even under adversarially chosen, time-varying input distributions, and provides algorithms, frameworks, and bounds for this setting.

Contribution

It extends the notion of replicability to adversarial online settings, develops algorithms for linear optimization and experts problems, and establishes regret bounds and lower bounds for such algorithms.

Findings

Developed adversarially replicable algorithms with sub-linear regret.

Created a framework to convert online learners into adversarially replicable algorithms.

Established regret lower bounds for replicable online algorithms.

Abstract

We investigate the concept of algorithmic replicability introduced by Impagliazzo et al. 2022, Ghazi et al. 2021, Ahn et al. 2024 in an online setting. In our model, the input sequence received by the online learner is generated from time-varying distributions chosen by an adversary (obliviously). Our objective is to design low-regret online algorithms that, with high probability, produce the exact same sequence of actions when run on two independently sampled input sequences generated as described above. We refer to such algorithms as adversarially replicable. Previous works (such as Esfandiari et al. 2022) explored replicability in the online setting under inputs generated independently from a fixed distribution; we term this notion as iid-replicability. Our model generalizes to capture both adversarial and iid input sequences, as well as their mixtures, which can be modeled by setting certain distributions as point-masses. We demonstrate adversarially replicable online learning algorithms for online linear optimization and the experts problem that achieve sub-linear regret. Additionally, we propose a general framework for converting an online learner into an adversarially replicable one within our setting, bounding the new regret in terms of the original algorithm's regret. We also present a nearly optimal (in terms of regret) iid-replicable online algorithm for the experts problem, highlighting the distinction between the iid and adversarial notions of replicability. Finally, we establish lower bounds on the regret (in terms of the replicability parameter and time) that any replicable online algorithm must incur.