Online Learning: Beyond Regret

TL;DR

This paper extends online learnability theory to a broad class of performance measures, identifying key quantities that determine learnability and improving upon previous results without focusing on specific algorithms.

Contribution

It generalizes the framework of online learnability to encompass various performance measures beyond regret, linking them through common complexity and convergence concepts.

Findings

Learnability depends on martingale convergence, future performance ability, and sequential Rademacher complexity.

The framework unifies multiple notions like internal regret, calibration, and approachability.

Results improve and extend previous bounds without relying on specific algorithms.

Abstract

We study online learnability of a wide class of problems, extending the results of (Rakhlin, Sridharan, Tewari, 2010) to general notions of performance measure well beyond external regret. Our framework simultaneously captures such well-known notions as internal and general Phi-regret, learning with non-additive global cost functions, Blackwell's approachability, calibration of forecasters, adaptive regret, and more. We show that learnability in all these situations is due to control of the same three quantities: a martingale convergence term, a term describing the ability to perform well if future is known, and a generalization of sequential Rademacher complexity, studied in (Rakhlin, Sridharan, Tewari, 2010). Since we directly study complexity of the problem instead of focusing on efficient algorithms, we are able to improve and extend many known results which have been previously derived via an algorithmic construction.