Relative Value Iteration for Stochastic Differential Games

TL;DR

This paper investigates zero-sum stochastic differential games, establishing existence and characterization of solutions to Isaac's equation under broad conditions, and analyzing the convergence of a relative value iteration scheme.

Contribution

It extends previous results by removing boundedness and geometric ergodicity assumptions, and studies convergence of relative value iteration in risk-sensitive control.

Findings

Existence of solutions to Isaac's equation under uniform stability.

Convergence of relative value iteration to elliptic Isaac's equation.

Extension of results to unbounded data and non-geometric ergodicity.

Abstract

We study zero-sum stochastic differential games with player dynamics governed by a nondegenerate controlled diffusion process. Under the assumption of uniform stability, we establish the existence of a solution to the Isaac's equation for the ergodic game and characterize the optimal stationary strategies. The data is not assumed to be bounded, nor do we assume geometric ergodicity. Thus our results extend previous work in the literature. We also study a relative value iteration scheme that takes the form of a parabolic Isaac's equation. Under the hypothesis of geometric ergodicity we show that the relative value iteration converges to the elliptic Isaac's equation as time goes to infinity. We use these results to establish convergence of the relative value iteration for risk-sensitive control problems under an asymptotic flatness assumption.