Strategizing against Learners in Bayesian Games

TL;DR

This paper analyzes the strategic interactions in Bayesian games where one player uses no-regret learning and the other aims to maximize utility, introducing new theoretical bounds and algorithms for optimal play.

Contribution

It establishes the minimum guaranteed payoff for the optimizer, designs algorithms to achieve this bound, and introduces the concept of polytope swap regret for analyzing such games.

Findings

The optimizer can guarantee a certain payoff regardless of the learner’s strategy.

Existence of learning algorithms that achieve the minimum guaranteed payoff.

Efficient implementation of these algorithms is possible.

Abstract

We study repeated two-player games where one of the players, the learner, employs a no-regret learning strategy, while the other, the optimizer, is a rational utility maximizer. We consider general Bayesian games, where the payoffs of both the optimizer and the learner could depend on the type, which is drawn from a publicly known distribution, but revealed privately to the learner. We address the following questions: (a) what is the bare minimum that the optimizer can guarantee to obtain regardless of the no-regret learning algorithm employed by the learner? (b) are there learning algorithms that cap the optimizer payoff at this minimum? (c) can these algorithms be implemented efficiently? While building this theory of optimizer-learner interactions, we define a new combinatorial notion of regret called polytope swap regret, that could be of independent interest in other settings.