DanZero: Mastering GuanDan Game with Reinforcement Learning

TL;DR

This paper introduces DanZero, an AI for the complex GuanDan card game, developed using reinforcement learning, distributed training, and self-play, achieving human-level performance after extensive training.

Contribution

First reinforcement learning-based AI for GuanDan, addressing large state/action space and variable player count with a distributed training framework.

Findings

DanZero outperforms 8 heuristic-based baseline AIs.

DanZero achieves human-level performance in gameplay.

Training completed after 30 days using 160 CPUs and 1 GPU.

Abstract

Card game AI has always been a hot topic in the research of artificial intelligence. In recent years, complex card games such as Mahjong, DouDizhu and Texas Hold'em have been solved and the corresponding AI programs have reached the level of human experts. In this paper, we are devoted to developing an AI program for a more complex card game, GuanDan, whose rules are similar to DouDizhu but much more complicated. To be specific, the characteristics of large state and action space, long length of one episode and the unsure number of players in the GuanDan pose great challenges for the development of the AI program. To address these issues, we propose the first AI program DanZero for GuanDan using reinforcement learning technique. Specifically, we utilize a distributed framework to train our AI system. In the actor processes, we carefully design the state features and agents generate samples by self-play. In the learner process, the model is updated by Deep Monte-Carlo Method. After training for 30 days using 160 CPUs and 1 GPU, we get our DanZero bot. We compare it with 8 baseline AI programs which are based on heuristic rules and the results reveal the outstanding performance of DanZero. We also test DanZero with human players and demonstrate its human-level performance.