Common Information Belief based Dynamic Programs for Stochastic Zero-sum Games with Competing Teams

TL;DR

This paper develops a framework using common information belief-based dynamic programming to analyze zero-sum stochastic games between competing teams with asymmetric information, providing bounds and strategies for such complex multi-agent scenarios.

Contribution

It introduces a novel CIB-based dynamic programming approach for zero-sum team games with asymmetric information, including bounds, conditions for game value, and strategies.

Findings

Bounds on game upper and lower values are derived.

Conditions identified where the game has a well-defined value.

An approximate dynamic programming method is proposed and illustrated.

Abstract

Decentralized team problems where players have asymmetric information about the state of the underlying stochastic system have been actively studied, but \emph{games} between such teams are less understood. We consider a general model of zero-sum stochastic games between two competing teams. This model subsumes many previously considered team and zero-sum game models. For this general model, we provide bounds on the upper (min-max) and lower (max-min) values of the game. Furthermore, if the upper and lower values of the game are identical (i.e., if the game has a \emph{value}), our bounds coincide with the value of the game. Our bounds are obtained using two dynamic programs based on a sufficient statistic known as the common information belief (CIB). We also identify certain information structures in which only the minimizing team controls the evolution of the CIB. In these cases, we show that one of our CIB based dynamic programs can be used to find the min-max strategy (in addition to the min-max value). We propose an approximate dynamic programming approach for computing the values (and the strategy when applicable) and illustrate our results with the help of an example.