Experts with Lower-Bounded Loss Feedback: A Unifying Framework

TL;DR

This paper introduces a unified feedback model for the best expert problem that generalizes full information and bandit feedback, providing optimal regret bounds and a second-order analysis framework.

Contribution

It proposes a new feedback model with lower bounds on expert losses, unifying and extending existing algorithms and regret analyses for bandit and full-information settings.

Findings

Proves near-optimal regret bounds for the new model.

Develops a second-order regret analysis framework involving Hessian-like expressions.

Connects the new model to graph-structured feedback settings.

Abstract

The most prominent feedback models for the best expert problem are the full information and bandit models. In this work we consider a simple feedback model that generalizes both, where on every round, in addition to a bandit feedback, the adversary provides a lower bound on the loss of each expert. Such lower bounds may be obtained in various scenarios, for instance, in stock trading or in assessing errors of certain measurement devices. For this model we prove optimal regret bounds (up to logarithmic factors) for modified versions of Exp3, generalizing algorithms and bounds both for the bandit and the full-information settings. Our second-order unified regret analysis simulates a two-step loss update and highlights three Hessian or Hessian-like expressions, which map to the full-information regret, bandit regret, and a hybrid of both. Our results intersect with those for bandits with graph-structured feedback, in that both settings can accommodate feedback from an arbitrary subset of experts on each round. However, our model also accommodates partial feedback at the single-expert level, by allowing non-trivial lower bounds on each loss.