PokerGPT: An End-to-End Lightweight Solver for Multi-Player Texas Hold'em via Large Language Model

TL;DR

PokerGPT introduces a lightweight, end-to-end LLM-based solver for multi-player Texas Hold'em, achieving high win rates with efficient training and interaction, surpassing prior methods in speed and size.

Contribution

This work presents the first lightweight LLM-based poker solver capable of multi-player Texas Hold'em, using reinforcement learning with human feedback and prompt engineering for effective decision-making.

Findings

PokerGPT outperforms previous approaches in win rate.

It requires less training time and smaller model size.

Provides fast and human-interactive decision advice.

Abstract

Poker, also known as Texas Hold'em, has always been a typical research target within imperfect information games (IIGs). IIGs have long served as a measure of artificial intelligence (AI) development. Representative prior works, such as DeepStack and Libratus heavily rely on counterfactual regret minimization (CFR) to tackle heads-up no-limit Poker. However, it is challenging for subsequent researchers to learn CFR from previous models and apply it to other real-world applications due to the expensive computational cost of CFR iterations. Additionally, CFR is difficult to apply to multi-player games due to the exponential growth of the game tree size. In this work, we introduce PokerGPT, an end-to-end solver for playing Texas Hold'em with arbitrary number of players and gaining high win rates, established on a lightweight large language model (LLM). PokerGPT only requires simple textual information of Poker games for generating decision-making advice, thus guaranteeing the convenient interaction between AI and humans. We mainly transform a set of textual records acquired from real games into prompts, and use them to fine-tune a lightweight pre-trained LLM using reinforcement learning human feedback technique. To improve fine-tuning performance, we conduct prompt engineering on raw data, including filtering useful information, selecting behaviors of players with high win rates, and further processing them into textual instruction using multiple prompt engineering techniques. Through the experiments, we demonstrate that PokerGPT outperforms previous approaches in terms of win rate, model size, training time, and response speed, indicating the great potential of LLMs in solving IIGs.