Comparison of Algorithms for Simple Stochastic Games (Full Version)

TL;DR

This paper compares three algorithms for simple stochastic games, introduces improvements including the first quadratic programming approach, and demonstrates significant speed-ups through extensive experiments.

Contribution

It provides a comprehensive comparison of value iteration, strategy iteration, and quadratic programming algorithms, including novel enhancements and the first implementation of quadratic programming for these games.

Findings

Quadratic programming can effectively solve simple stochastic games.

Improvements lead to significant speed-ups in algorithms.

First implementation of quadratic programming in PRISM-games.

Abstract

Simple stochastic games are turn-based 2.5-player zero-sum graph games with a reachability objective. The problem is to compute the winning probability as well as the optimal strategies of both players. In this paper, we compare the three known classes of algorithms -- value iteration, strategy iteration and quadratic programming -- both theoretically and practically. Further, we suggest several improvements for all algorithms, including the first approach based on quadratic programming that avoids transforming the stochastic game to a stopping one. Our extensive experiments show that these improvements can lead to significant speed-ups. We implemented all algorithms in PRISM-games 3.0, thereby providing the first implementation of quadratic programming for solving simple stochastic games.