Solving Large Extensive-Form Games with Strategy Constraints

TL;DR

This paper introduces a generalized Counterfactual Regret Minimization algorithm capable of efficiently computing optimal strategies in large extensive-form games with convex strategy constraints, addressing limitations of previous methods.

Contribution

It presents a novel algorithm that handles convex strategy constraints in large games, extending the applicability of equilibrium computation methods.

Findings

Effective risk mitigation in security games

Opponent modeling in poker with partial information

Algorithm scales to large extensive-form games

Abstract

Extensive-form games are a common model for multiagent interactions with imperfect information. In two-player zero-sum games, the typical solution concept is a Nash equilibrium over the unconstrained strategy set for each player. In many situations, however, we would like to constrain the set of possible strategies. For example, constraints are a natural way to model limited resources, risk mitigation, safety, consistency with past observations of behavior, or other secondary objectives for an agent. In small games, optimal strategies under linear constraints can be found by solving a linear program; however, state-of-the-art algorithms for solving large games cannot handle general constraints. In this work we introduce a generalized form of Counterfactual Regret Minimization that provably finds optimal strategies under any feasible set of convex constraints. We demonstrate the effectiveness of our algorithm for finding strategies that mitigate risk in security games, and for opponent modeling in poker games when given only partial observations of private information.