Best-of-Both-Worlds Algorithms for Partial Monitoring

TL;DR

This paper introduces the first algorithms that perform optimally in both stochastic and adversarial partial monitoring settings, with regret bounds tailored to game observability and complexity.

Contribution

It presents novel best-of-both-worlds algorithms for partial monitoring, with regret bounds for both stochastic and adversarial regimes based on game observability.

Findings

Regret bounds for non-degenerate locally observable games in stochastic and adversarial regimes.

Regret bounds for globally observable games in stochastic and adversarial regimes.

Algorithms are based on follow-the-regularized-leader with adaptive learning rates.

Abstract

This study considers the partial monitoring problem with $k$-actions and $d$-outcomes and provides the first best-of-both-worlds algorithms, whose regrets are favorably bounded both in the stochastic and adversarial regimes. In particular, we show that for non-degenerate locally observable games, the regret is $O(m^2 k^4 \log(T) \log(k_{\Pi} T) / \Delta_{\min})$ in the stochastic regime and $O(m k^{2/3} \sqrt{T \log(T) \log k_{\Pi}})$ in the adversarial regime, where $T$ is the number of rounds, $m$ is the maximum number of distinct observations per action, $\Delta_{\min}$ is the minimum suboptimality gap, and $k_{\Pi}$ is the number of Pareto optimal actions. Moreover, we show that for globally observable games, the regret is $O(c_{\mathcal{G}}^2 \log(T) \log(k_{\Pi} T) / \Delta_{\min}^2)$ in the stochastic regime and $O((c_{\mathcal{G}}^2 \log(T) \log(k_{\Pi} T))^{1/3} T^{2/3})$ in the adversarial regime, where $c_{\mathcal{G}}$ is a game-dependent constant. We also provide regret bounds for a stochastic regime with adversarial corruptions. Our algorithms are based on the follow-the-regularized-leader framework and are inspired by the approach of exploration by optimization and the adaptive learning rate in the field of online learning with feedback graphs.