Zero-Sum Games for Continuous-time Markov Decision Processes with Risk-Sensitive Average Cost Criterion

TL;DR

This paper studies zero-sum stochastic games for continuous-time Markov decision processes with risk-sensitive average costs, proving the existence of game value and saddle-point equilibria under stability conditions using advanced mathematical tools.

Contribution

It establishes the existence of a game value and saddle-point equilibrium for risk-sensitive continuous-time Markov games with unbounded rates, using a novel eigenpair approach.

Findings

Existence of the game value and saddle-point equilibrium.

Characterization of equilibrium via Hamilton-Jacobi-Isaacs equation.

Application to a controlled population system.

Abstract

We consider zero-sum stochastic games for continuous time Markov decision processes with risk-sensitive average cost criterion. Here the transition and cost rates may be unbounded. We prove the existence of the value of the game and a saddle-point equilibrium in the class of all stationary strategies under a Lyapunov stability condition. This is accomplished by establishing the existence of a principal eigenpair for the corresponding Hamilton-Jacobi-Isaacs (HJI) equation. This in turn is established by using the nonlinear version of Krein-Rutman theorem. We then obtain a characterization of the saddle-point equilibrium in terms of the corresponding HJI equation. Finally, we use a controlled population system to illustrate results.