# Compact Representation of Value Function in Partially Observable Stochastic Games

## TL;DR

This paper introduces a new compact belief representation and an algorithm that significantly improves the scalability of solving partially observable stochastic games, especially in large state spaces like security scenarios.

## Contribution

It proposes a novel low-dimensional belief abstraction technique and an associated algorithm that enhances scalability over existing methods.

## Key findings

- Dramatic scalability improvements demonstrated in experiments.
- Effective belief abstraction reduces computational complexity.
- Algorithm outperforms state-of-the-art in large-scale problems.

## Abstract

Value methods for solving stochastic games with partial observability model the uncertainty about states of the game as a probability distribution over possible states. The dimension of this belief space is the number of states. For many practical problems, for example in security, there are exponentially many possible states which causes an insufficient scalability of algorithms for real-world problems. To this end, we propose an abstraction technique that addresses this issue of the curse of dimensionality by projecting high-dimensional beliefs to characteristic vectors of significantly lower dimension (e.g., marginal probabilities). Our two main contributions are (1) novel compact representation of the uncertainty in partially observable stochastic games and (2) novel algorithm based on this compact representation that is based on existing state-of-the-art algorithms for solving stochastic games with partial observability. Experimental evaluation confirms that the new algorithm over the compact representation dramatically increases the scalability compared to the state of the art.