Playing Against Fair Adversaries in Stochastic Games with Total Rewards

TL;DR

This paper studies stochastic games with total rewards where the minimizer is fair, proving determinacy and optimal strategies, and presents a tool for their analysis with practical case studies.

Contribution

It introduces a new class of stochastic games with fair minimizers, proves their determinacy, and provides a method and tool for computing game values.

Findings

Games are determined with well-defined values.

Both players have memoryless, deterministic optimal strategies.

The approach is validated on case studies including UAV scenarios.

Abstract

We investigate zero-sum turn-based two-player stochastic games in which the objective of one player is to maximize the amount of rewards obtained during a play, while the other aims at minimizing it. We focus on games in which the minimizer plays in a fair way. We believe that these kinds of games enjoy interesting applications in software verification, where the maximizer plays the role of a system intending to maximize the number of "milestones" achieved, and the minimizer represents the behavior of some uncooperative but yet fair environment. Normally, to study total reward properties, games are requested to be stopping (i.e., they reach a terminal state with probability 1). We relax the property to request that the game is stopping only under a fair minimizing player. We prove that these games are determined, i.e., each state of the game has a value defined. Furthermore, we show that both players have memoryless and deterministic optimal strategies, and the game value can be computed by approximating the greatest-fixed point of a set of functional equations. We implemented our approach in a prototype tool, and evaluated it on an illustrating example and an Unmanned Aerial Vehicle case study.