AlphaDou: High-Performance End-to-End Doudizhu AI Integrating Bidding

TL;DR

This paper introduces AlphaDou, an AI that integrates bidding and cardplay for Doudizhu, achieving state-of-the-art performance through a modified reinforcement learning framework that handles the game's complexity.

Contribution

It presents a novel reinforcement learning approach that combines win rate and expectation estimation, enabling comprehensive Doudizhu gameplay including bidding and cardplay.

Findings

Achieved state-of-the-art performance in Doudizhu

Successfully integrated bidding and cardplay in AI

Demonstrated effectiveness of modified RL framework

Abstract

Artificial intelligence for card games has long been a popular topic in AI research. In recent years, complex card games like Mahjong and Texas Hold'em have been solved, with corresponding AI programs reaching the level of human experts. However, the game of Doudizhu presents significant challenges due to its vast state/action space and unique characteristics involving reasoning about competition and cooperation, making the game extremely difficult to solve.The RL model Douzero, trained using the Deep Monte Carlo algorithm framework, has shown excellent performance in Doudizhu. However, there are differences between its simplified game environment and the actual Doudizhu environment, and its performance is still a considerable distance from that of human experts. This paper modifies the Deep Monte Carlo algorithm framework by using reinforcement learning to obtain a neural network that simultaneously estimates win rates and expectations. The action space is pruned using expectations, and strategies are generated based on win rates. The modified algorithm enables the AI to perform the full range of tasks in the Doudizhu game, including bidding and cardplay. The model was trained in a actual Doudizhu environment and achieved state-of-the-art performance among publicly available models. We hope that this new framework will provide valuable insights for AI development in other bidding-based games.