Algorithms for zero-sum stochastic games with the risk-sensitive average criterion

TL;DR

This paper develops algorithms to compute approximate values and saddle points in zero-sum risk-sensitive average stochastic games with finite states and actions, supported by convergence proofs and a practical energy management example.

Contribution

It introduces the irreducibility coefficient, establishes its equivalence to irreducibility, and develops iterative algorithms for approximating values and saddle points.

Findings

Algorithms converge to $ ext{ε}$-approximations of the value.

Finite-step algorithm finds $ ext{ε}$-saddle points.

Numerical example demonstrates practical applicability.

Abstract

This paper is an attempt to compute the value and saddle points of zero-sum risk-sensitive average stochastic games. For the average games with finite states and actions, we first introduce the so-called irreducibility coefficient and then establish its equivalence to the irreducibility condition. Using this equivalence,we develop an iteration algorithm to compute $\varepsilon$-approximations of the value (for any given $\varepsilon>0$) and show its convergence. Based on $\varepsilon$-approximations of the value and the irreducibility coefficient, we further propose another iteration algorithm, which is proved to obtain $\varepsilon$-saddle points in finite steps. Finally, a numerical example of energy management in smart grids is provided to illustrate our results.