Computing Perfect Bayesian Equilibria, with Application to Empirical Game-Theoretic Analysis

TL;DR

This paper introduces a scalable algorithm for computing Perfect Bayesian Equilibria in two-player extensive-form games, improving the analysis of strategic interactions with imperfect information.

Contribution

It adapts Counterfactual Regret Minimization to efficiently compute PBE, with proven correctness for zero-sum games and demonstrated effectiveness in empirical game analysis.

Findings

Algorithm is correct for two-player zero-sum games.

PBE-CFR performs well on medium-to-large non-zero-sum EFGs.

Using PBE as a meta-strategy solver improves empirical game models.

Abstract

Perfect Bayesian Equilibrium (PBE) is a refinement of the Nash equilibrium for imperfect-information extensive-form games (EFGs) that enforces consistency between the two components of a solution: agents' strategy profile describing their decisions at information sets and the belief system quantifying their uncertainty over histories within an information set. We present a scalable approach for computing a PBE of an arbitrary two-player EFG. We adopt the definition of PBE enunciated by Bonanno in 2011 using a consistency concept based on the theory of belief revision due to Alchourr\'{o}n, G\"{a}rdenfors, and Makinson. Our algorithm for finding a PBE is an adaptation of Counterfactual Regret Minimization (CFR) that minimizes the expected regret at each information set given a belief system, while maintaining the necessary consistency criteria. We prove that our algorithm is correct for two-player zero-sum games and has a reasonable slowdown in time-complexity relative to classical CFR given the additional computation needed for refinement. We also experimentally demonstrate the competent performance of PBE-CFR in terms of equilibrium quality and running time on medium-to-large non-zero-sum EFGs. Finally, we investigate the effectiveness of using PBE for strategy exploration in empirical game-theoretic analysis. Specifically, we compute PBE as a meta-strategy solver (MSS) in a tree-exploiting variant of Policy Space Response Oracles (TE-PSRO). Our experiments show that PBE as an MSS leads to higher-quality empirical EFG models with complex imperfect information structures compared to MSSs based on an unrefined Nash equilibrium.