A Generic Strategy Improvement Method for Simple Stochastic Games

TL;DR

This paper introduces a generic strategy iteration algorithm for simple stochastic games, removing previous assumptions, providing complexity bounds, and unifying existing algorithms for improved efficiency and analysis.

Contribution

The paper presents a new generic strategy iteration algorithm that generalizes existing methods, removes the stopping assumption, and offers tighter complexity bounds for simple stochastic games.

Findings

GSIA is correct and more general than previous algorithms.

All known strategy iteration algorithms are special cases of GSIA.

The algorithm converges faster than existing methods, requiring fewer than r! iterations.

Abstract

We present a generic strategy iteration algorithm (GSIA) to find an optimal strategy of a simple stochastic game (SSG). We prove the correctness of GSIA, and derive a general complexity bound, which implies and improves on the results of several articles. First, we remove the assumption that the SSG is stopping, which is usually obtained by a polynomial blowup of the game. Second, we prove a tight bound on the denominator of the values associated to a strategy, and use it to prove that all strategy iteration algorithms are in fact fixed parameter tractable in the number of random vertices. All known strategy iteration algorithms can be seen as instances of GSIA, which allows to analyze the complexity of converge from below by Condon and to propose a class of algorithms generalising Gimbert and Horn's algorithm. These algorithms require less than $r!$ iterations in general and less iterations than the current best deterministic algorithm for binary SSGs given by Ibsen-Jensen and Miltersen.