Valet: A Standardized Testbed of Traditional Imperfect-Information Card Games

TL;DR

Valet is a comprehensive testbed of 21 traditional imperfect-information card games, standardized with RECYCLE, enabling robust benchmarking of AI algorithms across diverse game types and complexities.

Contribution

It introduces Valet, a diverse collection of card games with standardized rules in RECYCLE, facilitating comparative AI research and benchmarking in imperfect-information games.

Findings

Baseline Monte Carlo Tree Search performance data

Characterization of game branching factors and durations

Demonstration of Valet's suitability as a benchmarking suite

Abstract

AI algorithms for imperfect-information games are typically compared using performance metrics on individual games, making it difficult to assess robustness across game choices. Card games are a natural domain for imperfect information due to hidden hands and stochastic draws. To facilitate comparative research on imperfect-information game-playing algorithms and game systems, we introduce Valet, a diverse and comprehensive testbed of 21 traditional imperfect-information card games. These games span multiple genres, cultures, player counts, deck structures, mechanics, winning conditions, and methods of hiding and revealing information. To standardize implementations across systems, we encode the rules of each game in RECYCLE, a card game description language. We empirically characterize each game's branching factor and duration using random simulations, reporting baseline score distributions for a Monte Carlo Tree Search player against random opponents to demonstrate the suitability of Valet as a benchmarking suite.