Online Learning with Composite Loss Functions

TL;DR

This paper introduces a new class of online learning problems with composite loss functions, analyzing their minimax regret under bandit feedback and distinguishing between hard and easy problem types based on the loss function used.

Contribution

The paper characterizes the minimax regret for composite loss functions in online learning, revealing the complexity differences between min/max and linear combinations of recent adversarial values.

Findings

Minimax regret is  (T^{2/3}) for min/max functions.

Minimax regret is  ( ext{sqrt}(T)) for linear functions.

Problems with min/max functions are provably hard, similar to bandit with switching costs.

Abstract

We study a new class of online learning problems where each of the online algorithm's actions is assigned an adversarial value, and the loss of the algorithm at each step is a known and deterministic function of the values assigned to its recent actions. This class includes problems where the algorithm's loss is the minimum over the recent adversarial values, the maximum over the recent values, or a linear combination of the recent values. We analyze the minimax regret of this class of problems when the algorithm receives bandit feedback, and prove that when the minimum or maximum functions are used, the minimax regret is $\tilde \Omega(T^{2/3})$ (so called hard online learning problems), and when a linear function is used, the minimax regret is $\tilde O(\sqrt{T})$ (so called easy learning problems). Previously, the only online learning problem that was known to be provably hard was the multi-armed bandit with switching costs.