Hierarchical Decompositions of Stochastic Pursuit-Evasion Games

TL;DR

This paper introduces a hierarchical framework for efficiently solving large stochastic pursuit-evasion games by decomposing the environment into superstates, enabling scalable computation of Nash equilibria with maintained performance.

Contribution

The paper presents a novel hierarchical approach that combines local and aggregated pursuit-evasion games, reducing computational complexity in large grid worlds.

Findings

Significantly reduces computation time compared to flat Nash solutions.

Maintains competitive performance levels in large grid environments.

Effective in solving large-scale stochastic pursuit-evasion games.

Abstract

In this work we present a hierarchical framework for solving discrete stochastic pursuit-evasion games (PEGs) in large grid worlds. With a partition of the grid world into superstates (e.g., "rooms"), the proposed approach creates a two-resolution decision-making process, which consists of a set of local PEGs at the original state level and an aggregated PEG at the superstate level. Having much smaller cardinality, both the local games and the aggregated game can be easily solved to a Nash equilibrium. To connect the decision-making at the two resolutions, we use the Nash values of the local PEGs as the rewards for the aggregated game. Through numerical simulations, we show that the proposed hierarchical framework significantly reduces the computation overhead, while still maintaining a satisfactory level of performance when competing against the flat Nash policies.