Operator approach to values of stochastic games with varying stage duration

TL;DR

This paper investigates the relationship between stochastic game values with varying stage durations, their associated operators, and continuous-time limits, establishing convergence and independence results for different game frameworks.

Contribution

It provides new theoretical results linking discrete and continuous stochastic game values, showing convergence and invariance under varying stage durations.

Findings

Value converges as stage duration approaches zero.

Asymptotic behavior is independent of stage duration.

Existence of continuous-time game value as limit of discretizations.

Abstract

We study the links between the values of stochastic games with varying stage duration $h$, the corresponding Shapley operators $\bf{T}$ and ${\bf{T}}\_h$and the solution of $\dot f\_t = ({\bf{T}} - Id )f\_t$. Considering general non expansive maps we establish two kinds of results, under both the discounted or the finite length framework, that apply to the class of "exact" stochastic games. First, for a fixed length or discount factor, the value converges as the stage duration go to 0. Second, the asymptotic behavior of the value as the length goes to infinity, or as the discount factor goes to 0, does not depend on the stage duration. In addition, these properties imply the existence of the value of the finite length or discounted continuous time game (associated to a continuous time jointly controlled Markov process), as the limit of the value of any time discretization with vanishing mesh.