An $\epsilon$-Optimal Sequential Approach for Solving zs-POSGs

TL;DR

This paper introduces a sequential decision process framework for zs-POSGs that linearizes complex backups, reducing computational complexity and enabling effective policy extraction, thus solving previously intractable problems.

Contribution

It recasts zs-POSGs as sequential processes using new sufficient statistics, enabling polynomial-time updates and direct safe policy extraction, advancing practical solution methods.

Findings

Algorithms outperform state-of-the-art methods

Reduces update complexity from exponential to polynomial

Enables solving previously intractable domains

Abstract

While recent reductions of zero-sum partially observable stochastic games (zs-POSGs) to transition-independent stochastic games (TI-SGs) theoretically admit dynamic programming, practical solutions remain stifled by the inherent non-linearity and exponential complexity of the simultaneous minimax backup. In this work, we surmount this computational barrier by rigorously recasting the simultaneous interaction as a sequential decision process via the principle of separation. We introduce distinct sufficient statistics for valuation and execution, the sequential occupancy state and the private occupancy family, which reveal a latent geometry in the optimal value function. This structural insight allows us to linearise the backup operator, reducing the update complexity from exponential to polynomial while enabling the direct extraction of safe policies without heuristic bookkeeping. Experimental results demonstrate that algorithms leveraging this sequential framework significantly outperform state-of-the-art methods, effectively rendering previously intractable domains solvable.