Finite-Horizon Partially Observable Semi-Markov Games with Risk Probability Criteria

TL;DR

This paper develops a framework for analyzing finite-horizon partially observable semi-Markov games with a focus on risk probability criteria, establishing key theoretical results including the existence of Nash equilibria.

Contribution

It introduces a novel approach using augmented state space and derives the Shapley equation for these complex games, proving existence and uniqueness of the value function.

Findings

Established a comparison theorem for the game model

Derived the Shapley equation under the probability criterion

Proved the existence and uniqueness of the value function and Nash equilibrium

Abstract

This paper studies partially observable two-person zero-sum semi-Markov games under a probability criterion, in which the system state may not be completely observed. It focuses on the probability that the accumulated rewards of player 1 (i.e., the incurred costs of player 2) fall short of a specified target at the terminal stage, which represents the risk of player 1 and the capacity of player 2. We study the game model via the technology of augmenting state space with the joint conditional distribution of the current unobserved state and the remaining goal. Under a mild condition, we establish a comparison theorem and derive the Shapley equation for the probability criterion. As a consequence, we prove the existence and the uniqueness of the value function and the existence of a Nash equilibrium.