Nash Approximation Gap in Truncated Infinite-horizon Partially Observable Markov Games

TL;DR

This paper introduces a finite-memory truncation method to approximate infinite-horizon POMGs, ensuring near-equilibrium solutions with increasing truncation length under certain stability conditions.

Contribution

It proposes a novel finite-memory truncation framework for infinite-horizon POMGs and proves its effectiveness in approximating Nash equilibria.

Findings

Truncated game Nash equilibria are $ ext{epsilon}$-close to original game equilibria.

Under filter stability, approximation error diminishes as truncation length increases.

The framework makes infinite-horizon POMGs computationally tractable.

Abstract

Partially Observable Markov Games (POMGs) provide a general framework for modeling multi-agent sequential decision-making under asymmetric information. A common approach is to reformulate a POMG as a fully observable Markov game over belief states, where the state is the conditional distribution of the system state and agents' private information given common information, and actions correspond to mappings (prescriptions) from private information to actions. However, this reformulation is intractable in infinite-horizon settings, as both the belief state and action spaces grow with the accumulation of information over time. We propose a finite-memory truncation framework that approximates infinite-horizon POMGs by a finite-state, finite-action Markov game, where agents condition decisions only on finite windows of common and private information. Under suitable filter stability (forgetting) conditions, we show that any Nash equilibrium of the truncated game is an $\varepsilon$-Nash equilibrium of the original POMG, where $\varepsilon \to 0$ as the truncation length increases.