Generalized Quantal Response Equilibrium: Existence and Efficient Learning

TL;DR

This paper introduces Generalized Quantal Response Equilibrium (GQRE), a new solution concept for bounded rationality in finite games, along with an efficient learning algorithm that converges under certain conditions and performs well on complex game examples.

Contribution

The paper proposes GQRE, extending Quantal Response Equilibrium, and develops a computationally efficient no-regret learning algorithm with convergence guarantees for complex general-sum games.

Findings

GQRE exists under mild conditions.

The learning algorithm converges with noisy gradient estimates.

Method performs well on high-rank and multi-player games.

Abstract

We introduce a new solution concept for bounded rational agents in finite normal-form general-sum games called Generalized Quantal Response Equilibrium (GQRE) which generalizes Quantal Response Equilibrium~\citep{mckelvey1995quantal}. In our setup, each player maximizes a smooth, regularized expected utility of the mixed profiles used, reflecting bounded rationality that subsumes stochastic choice. After establishing existence under mild conditions, we present computationally efficient no-regret independent learning via smoothened versions of the Frank-Wolfe algorithm. Our algorithm uses noisy but correlated gradient estimates generated via a simulation oracle that reports on repeated plays of the game. We analyze convergence properties of our algorithm under assumptions that ensure uniqueness of equilibrium, using a class of gap functions that generalize the Nash gap. We end by demonstrating the effectiveness of our method on a set of complex general-sum games such as high-rank two-player games, large action two-player games, and known examples of difficult multi-player games.