Bayes' Bluff: Opponent Modelling in Poker

TL;DR

This paper introduces a Bayesian model for opponent modeling in poker, addressing uncertainties in game dynamics and strategies, and demonstrates effective response methods in simplified and full Texas hold'em.

Contribution

It presents a novel Bayesian framework for opponent modeling in poker, separating game and strategy uncertainties, with methods for inference and response in complex poker scenarios.

Findings

Effective opponent strategy inference using Bayesian methods

Successful application to simplified and full Texas hold'em

Improved response strategies based on posterior distributions

Abstract

Poker is a challenging problem for artificial intelligence, with non-deterministic dynamics, partial observability, and the added difficulty of unknown adversaries. Modelling all of the uncertainties in this domain is not an easy task. In this paper we present a Bayesian probabilistic model for a broad class of poker games, separating the uncertainty in the game dynamics from the uncertainty of the opponent's strategy. We then describe approaches to two key subproblems: (i) inferring a posterior over opponent strategies given a prior distribution and observations of their play, and (ii) playing an appropriate response to that distribution. We demonstrate the overall approach on a reduced version of poker using Dirichlet priors and then on the full game of Texas hold'em using a more informed prior. We demonstrate methods for playing effective responses to the opponent, based on the posterior.