No-Regret Strategy Solving in Imperfect-Information Games via Pre-Trained Embedding

TL;DR

This paper introduces Embedding CFR, a novel strategy-solving algorithm for imperfect-information games that uses pre-trained embeddings to better distinguish information sets, leading to faster convergence and improved performance.

Contribution

It proposes a new embedding-based approach for information set abstraction in imperfect-information games, outperforming traditional clustering methods in convergence speed and accuracy.

Findings

Faster exploitability convergence in poker experiments.

First to use pre-trained low-dimensional embeddings for information set abstraction.

Theoretically reduces cumulative regret in strategy solving.

Abstract

High-quality information set abstraction remains a core challenge in solving large-scale imperfect-information extensive-form games (IIEFGs)--such as no-limit Texas Hold'em--where the finite nature of spatial resources hinders solving strategies for the full game. State-of-the-art AI methods rely on pre-trained discrete clustering for abstraction, yet their hard classification irreversibly discards critical information: specifically, the quantifiable subtle differences between information sets--vital for strategy solving--thus compromising the quality of such solving. Inspired by the word embedding paradigm in natural language processing, this paper proposes the Embedding CFR algorithm, a novel approach for solving strategies in IIEFGs within an embedding space. The algorithm pre-trains and embeds the features of individual information sets into an interconnected low-dimensional continuous space, where the resulting vectors more precisely capture both the distinctions and connections between information sets. Embedding CFR introduces a strategy-solving process driven by regret accumulation and strategy updates in this embedding space, with supporting theoretical analysis verifying its ability to reduce cumulative regret. Experiments on poker show that with the same spatial overhead, Embedding CFR achieves significantly faster exploitability convergence compared to cluster-based abstraction algorithms, confirming its effectiveness. Furthermore, to our knowledge, it is the first algorithm in poker AI that pre-trains information set abstractions via low-dimensional embedding for strategy solving.