High Probability Bound for Cross-Learning Contextual Bandits with Unknown Context Distributions

TL;DR

This paper provides a high-probability regret bound for cross-learning contextual bandits with unknown context distributions, improving the understanding of algorithm performance beyond expected regret.

Contribution

It offers a novel high-probability analysis of Schneider and Zimmert's algorithm, utilizing new insights into epoch dependencies and refined martingale inequalities.

Findings

Achieves near-optimal high-probability regret bounds

Introduces new analysis techniques for epoch dependencies

Refines martingale inequalities for better bounds

Abstract

Motivated by applications in online bidding and sleeping bandits, we examine the problem of contextual bandits with cross learning, where the learner observes the loss associated with the action across all possible contexts, not just the current round's context. Our focus is on a setting where losses are chosen adversarially, and contexts are sampled i.i.d. from a specific distribution. This problem was first studied by Balseiro et al. (2019), who proposed an algorithm that achieves near-optimal regret under the assumption that the context distribution is known in advance. However, this assumption is often unrealistic. To address this issue, Schneider and Zimmert (2023) recently proposed a new algorithm that achieves nearly optimal expected regret. It is well-known that expected regret can be significantly weaker than high-probability bounds. In this paper, we present a novel, in-depth analysis of their algorithm and demonstrate that it actually achieves near-optimal regret with high probability. There are steps in the original analysis by Schneider and Zimmert (2023) that lead only to an expected bound by nature. In our analysis, we introduce several new insights. Specifically, we make extensive use of the weak dependency structure between different epochs, which was overlooked in previous analyses. Additionally, standard martingale inequalities are not directly applicable, so we refine martingale inequalities to complete our analysis.