Beyond Nash Equilibrium: Achieving Bayesian Perfect Equilibrium with Belief Update Fictitious Play

TL;DR

This paper introduces Belief Update Fictitious Play (BUFP), a novel method that extends traditional equilibrium concepts to Bayesian Perfect Equilibrium, improving strategic decision-making in incomplete information games.

Contribution

The paper proposes BUFP, a new approach blending fictitious play with belief updates to achieve BPE, surpassing traditional methods like CFR in complex game scenarios.

Findings

BUFP achieves convergence to BPE by adjusting iteration stepsizes.

BUFP outperforms CFR, with a 48.53% benefit increase in dominated strategy scenarios.

The method enhances AI decision-making by accounting for irrational opponent behaviors.

Abstract

In the domain of machine learning and game theory, the quest for Nash Equilibrium (NE) in extensive-form games with incomplete information is challenging yet crucial for enhancing AI's decision-making support under varied scenarios. Traditional Counterfactual Regret Minimization (CFR) techniques excel in navigating towards NE, focusing on scenarios where opponents deploy optimal strategies. However, the essence of machine learning in strategic game play extends beyond reacting to optimal moves; it encompasses aiding human decision-making in all circumstances. This includes not only crafting responses to optimal strategies but also recovering from suboptimal decisions and capitalizing on opponents' errors. Herein lies the significance of transitioning from NE to Bayesian Perfect Equilibrium (BPE), which accounts for every possible condition, including the irrationality of opponents. To bridge this gap, we propose Belief Update Fictitious Play (BUFP), which innovatively blends fictitious play with belief to target BPE, a more comprehensive equilibrium concept than NE. Specifically, through adjusting iteration stepsizes, BUFP allows for strategic convergence to both NE and BPE. For instance, in our experiments, BUFP(EF) leverages the stepsize of Extensive Form Fictitious Play (EFFP) to achieve BPE, outperforming traditional CFR by securing a 48.53\% increase in benefits in scenarios characterized by dominated strategies.