Prior-Agnostic Incentive-Compatible Exploration

TL;DR

This paper introduces a method for incentivizing agents to follow exploration strategies in bandit problems without requiring a common prior, using swap regret bounds to ensure approximate equilibrium in dynamic, multi-agent environments.

Contribution

It demonstrates that swap regret bounds can ensure incentive compatibility in dynamic bandit settings with conflicting priors, even without shared prior knowledge.

Findings

Swap regret bounds lead to approximate Bayes Nash equilibrium.

Agents with some uncertainty about their arrival time follow forecasts faithfully.

The approach applies to adaptive and weighted regret algorithms in bandit problems.

Abstract

In bandit settings, optimizing long-term regret metrics requires exploration, which corresponds to sometimes taking myopically sub-optimal actions. When a long-lived principal merely recommends actions to be executed by a sequence of different agents (as in an online recommendation platform) this provides an incentive misalignment: exploration is "worth it" for the principal but not for the agents. Prior work studies regret minimization under the constraint of Bayesian Incentive-Compatibility in a static stochastic setting with a fixed and common prior shared amongst the agents and the algorithm designer. We show that (weighted) swap regret bounds on their own suffice to cause agents to faithfully follow forecasts in an approximate Bayes Nash equilibrium, even in dynamic environments in which agents have conflicting prior beliefs and the mechanism designer has no knowledge of any agents beliefs. To obtain these bounds, it is necessary to assume that the agents have some degree of uncertainty not just about the rewards, but about their arrival time -- i.e. their relative position in the sequence of agents served by the algorithm. We instantiate our abstract bounds with concrete algorithms for guaranteeing adaptive and weighted regret in bandit settings.