Learning Recommender Mechanisms for Bayesian Stochastic Games

TL;DR

This paper introduces a bi-level reinforcement learning method to design recommender mechanisms in Bayesian stochastic games, effectively balancing social welfare and incentive compatibility.

Contribution

It proposes a novel neural network-based mechanism design approach tailored for Bayesian stochastic games, addressing limitations of previous methods.

Findings

Achieves social welfare comparable to cooperative multi-agent RL baselines.

Provides significantly better incentive properties than existing mechanisms.

Demonstrates effectiveness on multiple repeated and stochastic games.

Abstract

An important challenge in non-cooperative game theory is coordinating on a single (approximate) equilibrium from many possibilities - a challenge that becomes even more complex when players hold private information. Recommender mechanisms tackle this problem by recommending strategies to players based on their reported type profiles. A key consideration in such mechanisms is to ensure that players are incentivized to participate, report their private information truthfully, and follow the recommendations. While previous work has focused on designing recommender mechanisms for one-shot and extensive-form games, these approaches cannot be effectively applied to stochastic games, particularly if we constrain recommendations to be Markov stationary policies. To bridge this gap, we introduce a novel bi-level reinforcement learning approach for automatically designing recommender mechanisms in Bayesian stochastic games. Our method produces a mechanism represented by a parametric function (such as a neural network), and is therefore highly efficient at execution time. Experimental results on two repeated and two stochastic games demonstrate that our approach achieves social welfare levels competitive with cooperative multi-agent reinforcement learning baselines, while also providing significantly improved incentive properties.