$\epsilon$-Optimally Solving Zero-Sum POSGs

TL;DR

This paper introduces a novel operator leveraging uniform continuity properties to solve zero-sum POSGs more efficiently, significantly reducing the complexity of linear programs needed for solution improvements and enhancing scalability.

Contribution

It presents a new operator and point-based value iteration algorithms that improve scalability of solving zero-sum POSGs by reducing constraints in linear programs.

Findings

Reduced constraints in linear programs for solution improvement

Enhanced scalability of value iteration algorithms

Maintained optimality guarantees in various domains

Abstract

A recent method for solving zero-sum partially observable stochastic games (zs-POSGs) embeds the original game into a new one called the occupancy Markov game. This reformulation allows applying Bellman's principle of optimality to solve zs-POSGs. However, improving a current solution requires solving a linear program with exponentially many potential constraints, which significantly restricts the scalability of this approach. This paper exploits the optimal value function's novel uniform continuity properties to overcome this limitation. We first construct a new operator that is computationally more efficient than the state-of-the-art update rules without compromising optimality. In particular, improving a current solution now involves a linear program with an exponential drop in constraints. We then also show that point-based value iteration algorithms utilizing our findings improve the scalability of existing methods while maintaining guarantees in various domains.