Adaptive Oracle-Efficient Online Learning

TL;DR

This paper introduces an adaptive, oracle-efficient online learning algorithm that performs well in friendly environments like small-loss scenarios and IID data, overcoming previous limitations of non-adaptive methods.

Contribution

It presents a new framework for designing oracle-efficient algorithms that adapt to small-loss and IID environments, under a condition called approximability.

Findings

The algorithm performs well in small-loss environments.

The algorithm extends to IID data with a best-of-both-worlds guarantee.

Applicable to real-world settings like online auctions and classification.

Abstract

The classical algorithms for online learning and decision-making have the benefit of achieving the optimal performance guarantees, but suffer from computational complexity limitations when implemented at scale. More recent sophisticated techniques, which we refer to as oracle-efficient methods, address this problem by dispatching to an offline optimization oracle that can search through an exponentially-large (or even infinite) space of decisions and select that which performed the best on any dataset. But despite the benefits of computational feasibility, oracle-efficient algorithms exhibit one major limitation: while performing well in worst-case settings, they do not adapt well to friendly environments. In this paper we consider two such friendly scenarios, (a) "small-loss" problems and (b) IID data. We provide a new framework for designing follow-the-perturbed-leader algorithms that are oracle-efficient and adapt well to the small-loss environment, under a particular condition which we call approximability (which is spiritually related to sufficient conditions provided by Dud\'{i}k et al., [2020]). We identify a series of real-world settings, including online auctions and transductive online classification, for which approximability holds. We also extend the algorithm to an IID data setting and establish a "best-of-both-worlds" bound in the oracle-efficient setting.