Combining Prediction of Human Decisions with ISMCTS in Imperfect Information Games

TL;DR

This paper introduces SDMCTS, a novel algorithm that combines human decision prediction with ISMCTS to improve agent performance in imperfect information games, demonstrated through experiments in the Cheat Game.

Contribution

The paper presents SDMCTS, integrating predictive modeling of opponent actions into ISMCTS to enhance decision-making in imperfect information games against humans.

Findings

SDMCTS outperforms baseline agents in the Cheat Game.

Performance of SDMCTS improves with better predictive models.

Experimental results with 120 human players validate the approach.

Abstract

Monte Carlo Tree Search (MCTS) has been extended to many imperfect information games. However, due to the added complexity that uncertainty introduces, these adaptations have not reached the same level of practical success as their perfect information counterparts. In this paper we consider the development of agents that perform well against humans in imperfect information games with partially observable actions. We introduce the Semi-Determinized-MCTS (SDMCTS), a variant of the Information Set MCTS algorithm (ISMCTS). More specifically, SDMCTS generates a predictive model of the unobservable portion of the opponent's actions from historical behavioral data. Next, SDMCTS performs simulations on an instance of the game where the unobservable portion of the opponent's actions are determined. Thereby, it facilitates the use of the predictive model in order to decrease uncertainty. We present an implementation of the SDMCTS applied to the Cheat Game, a well-known card game, with partially observable (and often deceptive) actions. Results from experiments with 120 subjects playing a head-to-head Cheat Game against our SDMCTS agents suggest that SDMCTS performs well against humans, and its performance improves as the predictive model's accuracy increases.