Solving Football by Exploiting Equilibrium Structure of 2p0s Differential Games with One-Sided Information

TL;DR

This paper introduces a structural property of certain two-player differential games with asymmetric information, enabling scalable solution methods and demonstrating effectiveness on a complex football scenario with continuous actions.

Contribution

It proves that equilibrium strategies in these games concentrate on a limited set of prototypes, reducing complexity and improving learning efficiency in multiagent reinforcement learning.

Findings

Equilibrium strategies focus on at most I action prototypes.

Game tree complexity reduces from U^{2K} to I^K or (I+1)^K.

Successfully applied to a 22-player football game with continuous actions.

Abstract

For a two-player imperfect-information extensive-form game (IIEFG) with $K$ time steps and a player action space of size $U$, the game tree complexity is $U^{2K}$, causing existing IIEFG solvers to struggle with large or infinite $(U,K)$, e.g., differential games with continuous action spaces. To partially address this scalability challenge, we focus on an important class of 2p0s games where the informed player (P1) knows the payoff while the uninformed player (P2) only has a belief over the set of $I$ possible payoffs. Such games encompass a wide range of scenarios in sports, defense, cybersecurity, and finance. We prove that under mild conditions, P1's (resp. P2's) equilibrium strategy at any infostate concentrates on at most $I$ (resp. $I+1$) action prototypes. When $I\ll U$, this equilibrium structure causes the game tree complexity to collapse to $I^K$ for P1 when P2 plays best responses, and $(I+1)^K$ for P2 in a dual game where P1 plays best responses. We then show that exploiting this structure in model-free multiagent reinforcement learning and model predictive control leads to significant improvements in learning accuracy and efficiency from SOTA IIEFG solvers. Our demonstration solves a 22-player football game with continuous action spaces and $K=10$ time steps, where the offense team needs to strategically conceal their play until a critical moment in order to exploit information advantage. Code is available at https://github.com/ghimiremukesh/cams/tree/iclr