Matrix games with bandit feedback
Brendan O'Donoghue, Tor Lattimore, Ian Osband

TL;DR
This paper investigates zero-sum matrix games with unknown payoffs and bandit feedback, proposing new algorithms with regret guarantees that outperform existing methods, and highlighting the failure of Thompson sampling in this context.
Contribution
It introduces regret analyses for variants of UCB and K-learning tailored for matrix games with bandit feedback, applicable against any opponent, including adversarial ones.
Findings
UCB and K-learning variants achieve low regret in matrix games.
Thompson sampling performs poorly in this setting.
New algorithms outperform existing methods empirically.
Abstract
We study a version of the classical zero-sum matrix game with unknown payoff matrix and bandit feedback, where the players only observe each others actions and a noisy payoff. This generalizes the usual matrix game, where the payoff matrix is known to the players. Despite numerous applications, this problem has received relatively little attention. Although adversarial bandit algorithms achieve low regret, they do not exploit the matrix structure and perform poorly relative to the new algorithms. The main contributions are regret analyses of variants of UCB and K-learning that hold for any opponent, e.g., even when the opponent adversarially plays the best-response to the learner's mixed strategy. Along the way, we show that Thompson fails catastrophically in this setting and provide empirical comparison to existing algorithms.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Bandit Algorithms Research · Game Theory and Applications · Auction Theory and Applications
