Transformer Based Planning in the Observation Space with Applications to Trick Taking Card Games
Douglas Rebstock, Christopher Solinas, Nathan R. Sturtevant, Michael, Buro

TL;DR
This paper introduces GO-MCTS, a novel planning algorithm using transformers for generative modeling in the observation space, improving decision-making in imperfect information trick-taking card games.
Contribution
It presents a new MCTS variant that leverages transformers as generative models for observation sequences, tailored for imperfect information games.
Findings
GO-MCTS outperforms traditional methods in multiple card games.
Transformers effectively model observation sequences for planning.
Iterative training via self-play enhances the model's performance.
Abstract
Traditional search algorithms have issues when applied to games of imperfect information where the number of possible underlying states and trajectories are very large. This challenge is particularly evident in trick-taking card games. While state sampling techniques such as Perfect Information Monte Carlo (PIMC) search has shown success in these contexts, they still have major limitations. We present Generative Observation Monte Carlo Tree Search (GO-MCTS), which utilizes MCTS on observation sequences generated by a game specific model. This method performs the search within the observation space and advances the search using a model that depends solely on the agent's observations. Additionally, we demonstrate that transformers are well-suited as the generative model in this context, and we demonstrate a process for iteratively training the transformer via population-based self-play.…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsArtificial Intelligence in Games · AI-based Problem Solving and Planning · Intelligent Tutoring Systems and Adaptive Learning
