An accretive operator approach to ergodic zero-sum stochastic games

TL;DR

This paper investigates ergodicity in zero-sum stochastic games with finite states and unbounded payoffs, using operator theory and accretive mappings to characterize conditions for the existence of uniform values and optimal strategies.

Contribution

It introduces an operator-theoretical framework for ergodicity in stochastic games, generalizing classical ergodicity conditions through accretive mappings and geometric transition probability conditions.

Findings

Characterization of ergodicity via solvability of an optimality equation

Proof of the existence of uniform value and stationary strategies

Generalization of ergodicity conditions for Markov processes

Abstract

We study some ergodicity property of zero-sum stochastic games with a finite state space and possibly unbounded payoffs. We formulate this property in operator-theoretical terms, involving the solvability of an optimality equation for the Shapley operators (i.e., the dynamic programming operators) of a family of perturbed games. The solvability of this equation entails the existence of the uniform value, and its solutions yield uniform optimal stationary strategies. We first provide an analytical characterization of this ergodicity property, and address the generic uniqueness, up to an additive constant, of the solutions of the optimality equation. Our analysis relies on the theory of accretive mappings, which we apply to maps of the form $Id - T$ where $T$ is nonexpansive. Then, we use the results of a companion work to characterize the ergodicity of stochastic games by a geometrical condition imposed on the transition probabilities. This condition generalizes classical notion of ergodicity for finite Markov chains and Markov decision processes.