Zero-sum continuous-time Markov games with one-side stopping

TL;DR

This paper studies a zero-sum continuous-time Markov game with control and stopping, deriving a system of ODEs with unilateral constraints to determine optimal strategies and proving existence and uniqueness of solutions.

Contribution

It introduces a novel approach to analyze control and stopping in Markov games via a system of constrained ODEs, establishing existence and uniqueness results.

Findings

Derived a Bellman-type system of ODEs with unilateral constraints.

Proved existence and uniqueness of solutions to the constrained ODE system.

Provided a method to obtain optimal strategies from the ODE solutions.

Abstract

The paper is concerned with a variant of the continuous-time finite state Markov game of control and stopping where both players can affect transition rates, while only one player can choose a stopping time. We use the dynamic programming principle and reduce this problem to a system of ODEs with unilateral constraints. This system plays the role of the Bellman equation. We show that its solution provides the optimal strategies of the players. Additionally, we prove the existence and uniqueness theorem for the deduced system of ODEs with unilateral constraints.