Preference-CFR$\:$ Beyond Nash Equilibrium for Better Game Strategies

TL;DR

This paper introduces Preference-CFR, a new algorithm that enhances game AI by enabling strategic diversity and adaptability to different play styles, beyond traditional Nash equilibrium-focused methods.

Contribution

Preference-CFR incorporates preference and vulnerability parameters to diversify strategies, improving AI adaptability and generating novel, distinct play styles in imperfect information games.

Findings

Successfully trained aggressive and loose passive strategies in Texas Hold'em

Strategies match performance of traditional CFR but with distinct behavioral patterns

Produces strategies that diverge from conventional heuristics, offering new insights

Abstract

Artificial intelligence (AI) has surpassed top human players in a variety of games. In imperfect information games, these achievements have primarily been driven by Counterfactual Regret Minimization (CFR) and its variants for computing Nash equilibrium. However, most existing research has focused on maximizing payoff, while largely neglecting the importance of strategic diversity and the need for varied play styles, thereby limiting AI's adaptability to different user preferences. To address this gap, we propose Preference-CFR (Pref-CFR), a novel method that incorporates two key parameters: preference degree and vulnerability degree. These parameters enable the AI to adjust its strategic distribution within an acceptable performance loss threshold, thereby enhancing its adaptability to a wider range of strategic demands. In our experiments with Texas Hold'em, Pref-CFR successfully trained Aggressive and Loose Passive styles that not only match original CFR-based strategies in performance but also display clearly distinct behavioral patterns. Notably, for certain hand scenarios, Pref-CFR produces strategies that diverge significantly from both conventional expert heuristics and original CFR outputs, potentially offering novel insights for professional players.