No-Regret Learning in Extensive-Form Games with Imperfect Recall

TL;DR

This paper extends counterfactual regret minimization (CFR) to games with imperfect recall, providing the first regret bounds for such scenarios and demonstrating practical memory reductions in various game domains.

Contribution

It introduces the first regret bounds for CFR in imperfect recall games and shows how to apply these bounds to full games via abstractions.

Findings

Regret bounds are established for CFR in imperfect recall games.

Imperfect recall can significantly reduce memory usage with minimal regret increase.

Theoretical results are validated in die-roll poker, phantom tic-tac-toe, and Bluff.

Abstract

Counterfactual Regret Minimization (CFR) is an efficient no-regret learning algorithm for decision problems modeled as extensive games. CFR's regret bounds depend on the requirement of perfect recall: players always remember information that was revealed to them and the order in which it was revealed. In games without perfect recall, however, CFR's guarantees do not apply. In this paper, we present the first regret bound for CFR when applied to a general class of games with imperfect recall. In addition, we show that CFR applied to any abstraction belonging to our general class results in a regret bound not just for the abstract game, but for the full game as well. We verify our theory and show how imperfect recall can be used to trade a small increase in regret for a significant reduction in memory in three domains: die-roll poker, phantom tic-tac-toe, and Bluff.